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1  Introduction 

The  development  and  implementation  of  advanced  aerospace  vehicles  is  an  endeavor  that  can  potentially 
affect  long  term  aviation  operations  and  future  system  capabilities  for  several  decades.  Selecting  the  best 
vehicle  configuration(s)  requires  a  thorough  understanding  of  the  capabilities  and  life-cycle  considerations 
required  by  the  end  user,  the  vehicle's  full  spectrum  operations,  as  well  as  technologies  impacting  both 
operational  needs  and  system  performance.  The  fundamental  goal  of  the  proposed  effort  involves  using 
the  Aerospace  Systems  Design  Laboratory  (ASDL)  established  expertise  in  the  fields  of  decision  support 
and  advanced  vehicle  modeling  and  simulation  (M&S)  to  develop  an  innovative  trade-off  environment  for 
advanced  vehicle  concepts  exploration. 

Over  the  span  from  October  2010  to  September  of  2016,  a  Capability  Assessment  and  Trade-off 
Environment  (CATE)  with  accompanying  Excel  tool  was  developed.  The  environment  is  powered  by 
surrogate  models  created  from  the  NASA  Design  and  Analysis  of  Rotorcraft  (NDARC)  code.  The  surrogate 
models  were  created  from  data  obtained  through  experiments  performed  in  NDARC  using  candidate  Joint 
Multi-Role  Rotorcraft  configurations  (Single  Main  Rotor,  Compound,  and  Tilt-rotor).  The  use  of  surrogates 
for  distinct  concept  families  provides  a  novel  way  of  doing  rapid  trades  to  investigate  how  performance 
and  vehicle  unit  cost  vary  across  the  different  designs.  To  assess  technology  impacts  on  vehicle 
capabilities,  CATE  includes  an  Interactive  Reconfigurable  Matrix  of  Alternatives  (IRMA)  that  allows  for 
input  and  management  of  technologies.  CATE  uses  Quality  Function  Deployment  (QFD)  style  qualitative 
analysis  for  technologies  that  do  not  necessarily  affect  mission  performance  but  do  affect  mission 
effectiveness.  Users  can  assess  technologies  by  manually  selecting  options  using  the  IRMA  or  by  using  a 
genetic  algorithm  to  perform  a  selection  based  on  the  user's  objectives  [1], 

This  fiscal  year  work  aimed  to  extend  the  capabilities  that  currently  exist  in  CATE.  To  increase  the  fidelity 
of  the  results  in  CATE,  a  comprehensive  rotor  performance  analysis  using  RCAS  (Rotor  Comprehensive 
Analysis  System)  has  been  used  to  calibrate  a  new  NDARC  model  that  is  then  integrated  directly  into  CATE. 
To  increase  the  accuracy  of  the  calibration,  an  optimization  algorithm  has  been  wrapped  around  Wayne 
Johnson's  Rotor  Performance  Spreadsheet,  varying  the  available  NDARC  variables  to  best  match  the 
calibration  data.  This  process  provides  a  quick  and  efficient  way  to  calibrate  CATE  to  new  models, 
increasing  the  tools  flexibility  and  accuracy. 
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To  improve  the  capabilities  of  the  IRMA  in  CATE,  an  extensive  rotorcraft  technology  literature  search  was 
performed  in  order  to  capture  new  rotorcraft  technologies.  During  the  literature  search,  different 
technologies  were  identified,  along  with  their  impacts  on  the  various  components  of  the  rotorcraft  (i.e. 
physical/functional).  These  impacts  were  then  modeled  in  the  CATE  environment  through  the  use  of  tech 
factors  on  NDARC  parameters.  This  work  ultimately  allows  for  new  technologies  to  be  rapidly  assessed  on 
a  baseline  architecture. 

In  order  to  extend  the  actual  modeling  capabilities,  investigation  on  how  OpenMDAO  can  be  used  to  solve 
Multidisciplinary  Design  Analysis  and  Optimization  (MDAO)  problems  was  performed.  The  open  source 
software  was  evaluated  as  a  mean  to  interface  with  NDARC  and  perform  calculations  on  the  results. 

The  capabilities  of  CATE  were  demonstrated  for  an  existing  vehicle,  the  UH-60  Black  Hawk.  First,  a  new 
procedure  to  calibrate  NDARC  files  was  illustrated  for  the  UH-60Aand  UH-60L.  The  power  required,  power 
available  and  component  weights  were  calibrated  with  published  data.  Technologies  were  implemented 
on  the  vehicle  model  and  the  performance  and  sizing  impacts  were  derived.  Among  them,  the 
technologies  used  to  perform  the  UH-60Lto  UH-60M  upgraded  were  implemented  and  the  characteristics 
of  the  derived  UH-60M  were  analyzed. 

The  use  of  an  integrated  discrete  event  simulation  model  to  estimate  Reliability,  Availability, 
Maintainability  (RAM  for  rapid  system  trade-off  analysis  will  be  illustrated.  The  use  of  discrete  event 
simulation  tool  is  essential  to  this  method  as  it  enables  designers  to  evaluate  different  concepts  to  achieve 
a  desired  Operational  Availability  (Ao)  and  affordability. 
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2  Report  on  Work  Completed 


2.1  Rotor  Performance  Spreadsheet  Updates 

This  section  describes  the  work  done  to  integrate  higher  fidelity  rotor  analysis  capability  to  CATE.  Rotor 
Comprehensive  Analysis  System  (RCAS)  is  used  to  perform  this  analysis  and  the  results  are  presented  in 
the  following  sub-sections. 

2.1.1  Rotor  Performance  Analysis  with  RCAS 

As  the  first  attempt  at  the  integration  of  higher  fidelity  analysis  capabilities  into  the  CATE,  a 
comprehensive  rotor  performance  analysis  using  RCAS  has  been  performed  and  connected  to  CATE,  as 
shown  in  Figure  1  .  The  integration  of  RCAS  into  the  CATE  environment  is  carried  out  in  three  steps.  First, 
a  performance  sweep  of  blade  loading  and  advance  ratio  is  run  in  RCAS  to  obtain  the  rotor  induced  and 
profile  power  required  during  flight.  Next,  an  optimization  technique  is  used  to  create  an  NDARC  model 
by  calibrating  a  set  of  NDARC  variables  to  match  the  results  from  the  RCAS  models.  This  NDARC  model  is 
then  used  within  the  CATE  spreadsheet  to  obtain  higher  fidelity  performance  results  in  a  computationally 
efficient  approach. 


RCAS 

(Rotorcraft  Comprehensive 
Analysis  System) 


Figure  1  Integration  Flow  of  the  RCAS  and  the  CATE 


RCAS  results  were  obtained  for  both  hover  and  forward  flight  conditions  using  the  standard  UH-60A  blade 
configuration  with  SC1095  and  SC1094RB  airfoils.  The  RCAS  analysis  option  used  in  the  study  is  a  single 
blade  analysis  with  dynamic  inflow  rotor  model  including  dynamic  stall  and  compressibility  effects.  The 
hover  analyses  was  conducted  at  both  sea  level  static,  and  4,000ft  95F  hot  day  conditions,  while  sweeping 
the  blade  loading  between  values  of  0.07  ~  0.17,  which  corresponded  to  a  gross  weight  of  13,188  lbs  to 
32,018  lbs.  Forward  flight  analyses  was  performed  only  at  sea  level  static  conditions,  with  the  advance 
ratio  being  varied  from  0.116  to  0.419,  corresponding  to  flight  speeds  between  50  kts  to  180  kts. 
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The  RCAS  rotor  performance  results  were  compared  with  the  CAMRAD  II  results  (which  show  good 
correlation  with  the  UH-60  flight  test  data  [2])  in  Figure  2.  The  RCAS  hover  results  match  well  with  the 
CAMRAD  II  results  for  both  sea  level  static  and  the  4,000ft  95F  flight  conditions  in  terms  of  total  rotor 
power  as  well  as  the  induced  and  profile  component  power  values.  The  CAMRAD  II  results  are  included  in 
the  rotor  spreadsheet  provided  with  the  NDARC  package. 


Figure  2  Rotor  Hover  Performance  Comparison  at  SLS/4K95F  (RCAS  :  CAMRAD  II) 


However,  the  forward  flight  results  show  discrepancies  in  induced  and  profile  component  power  trends 
even  though  the  total  rotor  power  results  are  close  to  each  other,  as  shown  in  Figure  3.  This  gap  seems  to 
result  from  the  difference  in  the  rotor  inflow  option  between  the  free  wake  model  in  the  CAMRAD  II  and 
the  dynamic  inflow  model  in  RCAS,  and  requires  a  further  investigation. 


3000 


2500 


500 


Forward  Flight  HP  @  SLS 


Advance  Ratio 


Forward  Flight  HP  @  SLS 


Advance  Ratio 


Figure  3  Rotor  Forward  Flight  Performance  Comparison  at  SLS  (RCAS  :  CAMRAD  II) 
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Due  to  these  results,  the  rotor  spreadsheet  optimization  task  in  this  study  has  been  performed  with  the 
RCAS  hover  results  and  the  CAMRAD  II  forward  flight  results  combined.  The  optimized  calibration 
approach,  which  is  explained  in  more  detail  in  the  following  section,  demonstrated  the  ability  to  quickly 
and  accurately  calibrate  a  NDARC  model  to  match  the  higher  fidelity  RCAS/CAMRAD-II  data.  Using  the 
calibrated  NDARC  model,  the  new  sizing  results  showed  less  converged  weight  compared  to  the  current 
calibrated  model.  The  reason  has  been  investigated  and  found  to  be  due  to  less  hover  power  predictions 
in  the  new  model,  which  could  explain  the  vertical  climb  rate  difference  in  the  calibration  model.  CATE 
and  NDARC  investigations  using  the  optimized  variables  will  be  continued  and  reported  on  in  more  details 
in  the  next  year  study.  A  more  accurate  UH-60A  calibration  model  is  expected  to  be  obtained  through  this 
further  investigation. 

2.1.2  Optimization  of  NDARC  Rotor  Spreadsheet  Calibration 

The  purpose  of  the  NDARC  Spreadsheet  is  to  calibrate  a  set  of  NDARC  variables  to  match  higher  fidelity 
models  from  CAMRAD/RCAS  for  various  flight  conditions  (considering  both  hover  and  forward  flight).  The 
calibration  aims  to  minimize  the  overall  error  between  the  NDARC  predictions  and  higher  fidelity  models 
at  all  of  the  specified  flight  conditions.  Currently,  the  process  requires  the  user  to  manually  perform 
iterations  by  changing  the  NDARC  design  variables,  one  at  a  time,  until  they  are  satisfied  that  the  NDARC 
model  approximates  the  higher  fidelity  data  accurately  enough.  This  leads  to  ambiguity  in  the  results,  as 
there  is  currently  no  direct  way  to  quantity  the  accuracy  of  the  results;  rather,  users  rely  on  visually 
matching  five  graphs  to  determine  if  the  curve  fits  from  NDARC  are  matching  the  calibration  data. 
Additionally,  this  process  relies  heavily  on  the  user  having  knowledge  on  what  appropriate  values  are  for 
each  of  the  design  variables,  and  it  severely  restricts  the  exploration  of  the  design  space  (made  up  of  the 
different  combinations  of  NDARC  variables),  as  the  manual  iteration  will  likely  hone  in  on  a  single  local 
minimum,  rather  than  finding  the  best  global  solution  to  minimizing  the  error.  Finally,  the  use  of  manual 
iteration  to  perform  this  task  is  incredibly  inefficient,  especially  if  the  task  is  repeated  many  times  for  a 
different  set  of  calibration  data. 

To  address  the  issues  stated  above,  the  calibration  of  the  NDARC  variables  was  treated  as  a  multi-objective 

optimization  problem.  Using  the  process  described  in  Section  2. 1.1.3,  the  accuracy  of  the  calibration  is 

measured  using  two  values:  the  error  in  forward  flight  and  hover  conditions.  Minimizing  the  errors  in  both 

forward  flight  and  hover  simultaneously  presents  conflicting  design  objectives;  reducing  the  error  in 

forward  flight  creates  a  greater  error  in  hover,  and  vice  versa.  Thus,  there  is  not  one  single  calibration 
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setting  that  is  better  than  all  others.  Rather,  the  solution  to  this  problem  is  a  set  of  non-dominated  designs, 
having  the  property  that  the  performance  in  one  objective  cannot  be  improved  without  worsening  the 
performance  of  the  other  objective.  This  set  of  solutions  to  the  optimization  problem  is  referred  to  as  the 
"Pareto  Frontier",  an  example  of  which  is  provided  in  Figure  4.  The  selection  of  the  "best  design"  is  the 
one  that  represents  the  best  compromise  between  forward  flight  and  hover  conditions  based  on  some 
preference  by  the  user,  which  requires  a  multi-objective  decision  making  (MODM)  technique.  This  is 
achieved  through  the  use  of  the  NSGA-II  (Non-Dominated  Sorting  Genetic  Algorithm  II)  optimization 
algorithm,  as  described  in  the  following  sections. 


M 

§  1 
u-i  c 

ai  E 


Feasible  Design 
*  Space*  • 


/  ’ 


Pareto 

Frontier 


Forward  Flight  Error 
(Obj:  minimize) 


Figure  4  Pareto  Frontier  demonstrating  the  tradeoff  in  optimality  between  minimizing  error  in  forward  flight  and  hover 


2.1.2.1  Optimization  Overview 

The  use  of  optimization  techniques  to  automate  the  calibration  of  the  NDARC  spreadsheet  relies  on  two 
things:  the  set  of  NDARC  design  variables  is  fixed  and  known  to  the  user,  and  the  calibration  data  set  is 
known  and  can  be  provided  in  some  structured  format.  With  this  information,  enough  structure  is 
provided  to  allow  the  entire  process  to  be  automated,  requiring  minimal  user  set  up  while  providing  fast, 
accurate  results  given  that  the  information  provided  is  appropriate.  An  overview  of  the  new  calibration 
process  is  provided  in  Figure  5,  which  requires  four  steps: 


1.  In  the  NDARC  Calibration  spreadsheet,  select  which  NDARC  variables  are  design  variables  for  the 
optimization  process  versus  constant  parameters,  and  set  the  calibration  data 

2.  Run  the  NSGA-II  optimization  algorithm 

3.  Load  the  Pareto  Frontier  of  the  calibration  data  set  into  the  NDARC  spreadsheet 

4.  Use  a  multi-objective  decision  making  technique  (TOPSIS)  to  select  best  compromise  design  based 
on  the  user  preference  of  minimizing  error  in  forward  flight  versus  hover 
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It  is  likely  that  the  user  will  have  to  iterate  on  this  process  by  setting  different  bounds  or  values  for  the 
design  variables,  or  even  changing  which  variables  will  be  optimized  and  which  will  be  held  constant. 
However,  the  process  is  designed  such  that  these  iterations  can  be  done  rapidly,  requiring  the  user  to 
simply  set  the  new  design  variables  and  corresponding  values,  then  click  a  button  to  run  the  optimization 
and  load  all  of  the  results.  The  run  time  is  expected  to  be  on  the  order  of  1-2  minutes,  but  may  vary 
depending  on  the  number  of  design  variables  selected. 


Select  Design  Variables 
&  Set  Calibration  Data 


V 

NSGA-II 

Optimization 

I 


Load  Pareto 


Forward  Flight  Error 
(Obj:  minimize) 


Figure  5  Automation  of  calibration  process  using  optimization  technique 


2.1.2.2  Requirements  to  Run  Optimization  Process  Properly 

The  calibration  process  is  fully  automated  through  the  NDARC  spreadsheet,  requiring  minimal  effort  for 
the  user  to  set  up  the  optimization  problem.  However,  the  implementation  relies  on  a  structured  set  of 
information  in  this  spreadsheet  that  MUST  be  followed  exactly.  Additionally,  the  optimization  process 
utilizes  a  Python  code  that  has  been  compiled  into  an  executable  file,  called 
"RunNDARC_Optimization.exe".  This  executable,  along  with  the  required  Python  modules,  must  be 
located  in  a  folder  called  “NDARC  Optimization".  The  "NDARC  Optimization"  folder  must  be  in  the  same 
folder  as  the  NDARC  Spreadsheet  in  order  for  the  automation  process  to  work.  An  overview  of  each  step 
in  the  calibration  process  is  provided  below. 
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2.1.2.3  Calculation  of  Calibration  Accuracy 

In  the  Rotor  Performance  Spreadsheet,  the  NDARC  model  is  calibrated  to  match  the  five  flight  conditions 
listed  below.  The  first  three  flight  conditions  are  associated  with  forward  flight,  while  the  last  two  are 
associated  with  the  hover  flight  conditions.  To  reduce  the  dimensionality  of  the  Pareto  Frontier  of  the 

design  space,  the  errors  associated  with  each  of  these  five  parameters  were  reduced  to  obtain  a  single 

error  value  for  forward  flight,  and  a  single  value  for  hover. 

1.  Calculate  drag  coefficient  in  forward  flight  at  a  fixed  tip  Mach  number 

2.  Calculate  the  induced  power  coefficient  (kappa)  in  forward  flight  at  a  fixed  tip  Mach  number 

3.  Calculate  the  drag  coefficient  in  forward  flight  at  a  fixed  blade  loading  (CT/s) 

4.  Calculate  the  drag  coefficient  in  hover  at  a  fixed  tip  Mach  number 

5.  Calculate  the  induced  power  coefficient  in  hover  at  a  fixed  tip  Mach  number 

To  do  this,  for  each  of  the  five  parameters  the  error  is  calculated  as  the  sum  of  the  squared  relative  error, 
where  the  value  estimated  form  the  curve  fits  is  measured  relative  to  the  actual  value  provided  by  either 
the  higher  fidelity  analysis  tool  (such  as  RCAS)  or  actual  experimental  data.  This  is  shown  below  in  Equation 
1,  where  the  errors  are  calculated  for  the  drag  coefficient  and  induced  power  coefficient  in  forward  flight 
at  a  fixed  tip  Mach  number  (indicated  by  the  "Mtip"  subscript).  This  provides  five  error  values,  one  for 
each  of  the  parameters  listed  above. 


(FF  CD  Error') Mtip  =  Ef=1  VF  k  Error)mip  =  Ef=1  ^ ^ 

Equation  1 

The  final  error  for  the  forward  flight  and  hover  flight  conditions  is  then  calculated  by  taking  the  norm  of 
the  errors  associated  with  each  flight  condition,  as  shown  by  Equation  2.  This  approach  allows  for  the 
calibration  of  the  NDARC  models  to  be  measured  by  two  error  values,  rather  than  a  five  error  values 
associated  with  each  of  the  parameters  being  calculated  by  the  Rotor  Performance  Spreadsheet.  This 
simplifies  the  analysis  for  the  user,  and  makes  the  Pareto  Frontier  easier  to  visualize. 

FF  Error  =  Norm[(FF  CD  Error)mip,  (FF  k  Error)mip,  (FF  CD  Error) CT /s] 

Hover  Error  =  Norm[Hover  CD  Error,  Hover  k  Error ] 


Equation  2 
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2.1.2 A  NDARC Spreadsheet  User  Interfaces 


Set  Calibration  Data 

The  calibration  data  is  set  separately  on  the  "Calibration  Data  Sets"  sheet  of  the  NDARC  Excel  spreadsheet. 
Three  sets  of  data  are  required  for  the  calibration  process,  as  shown  by  Figure  6,  while  the  calibration 
variables  are  briefly  described  in  Table  1.  This  data  is  specific  to  the  curve  fits  developed  in  the  NDARC 
spreadsheet,  and  is  used  to  calibrate  the  NDARC  variables  to  match  the  five  flight  conditions  discussed  in 
the  previous  section. 


Forward  Flight  Calibration  Data 


Hover  Calibration  Data 


Figure  6  Calibration  data  tables  used  to  structure  the  information  for  the  optimization  code 


Table  1  Description  of  variables  required  in  the  calibration  data  set 


Variable 

Description 

mux 

Advance  ratio  along  the  x-axis 

muz 

Advance  ratio  along  the  z-axis 

CT/s 

Blade  Loading  (thrust  coefficient  /  solidity) 

MAT 

Maximum  Mach  number  at  the  advancing  tip 

Mtip 

Blade  tip  Mach  number 

Actual  Cd 

Actual  drag  coefficient 

Actual  Kappa 

Actual  induced  power  coefficient 

The  optimization  code  is  written  to  pull  the  data  out  of  these  specific  tables.  The  tables  can  be  of  arbitrary 
length  (the  code  will  dynamically  read  the  calibration  tables  until  it  has  found  the  last  row  with  data  in  it), 
but  the  column  order  MUST  be  followed  exactly.  Additionally,  the  first  column  of  each  table  has  a  label 
for  Case  number.  This  is  required  in  order  to  separate  the  data  displayed  on  the  graphs  in  the  "TOPSIS" 
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sheet,  so  the  user  must  take  care  to  separate  the  data  properly  into  different  cases.  For  example,  in  the 
"Fixed  Mtip"  table,  the  case  numbers  differentiate  between  the  data  that  pertain  to  different  CT/s  values. 


The  implementation  of  the  calibration  data  set  in  this  manner  adds  flexibility  to  the  spreadsheet,  as  the 
user  can  now  quickly  change  the  calibration  data  set  and  run  the  optimization  with  very  little  effort. 
Flowever,  the  code  is  limited  to  calibration  data  in  this  specific  format.  If  for  any  reason  the  type  of 
calibration  data  must  be  changed  (e.g.  no  longer  calculating  values  for  drag,  but  some  other  parameter), 
both  the  NDARC  spreadsheet  and  the  optimization  code  will  have  to  be  altered  to  reflect  this. 


Setting  Design  Variables 

The  user  interaction  required  to  set  up  and  run  the  optimization  is  contained  within  the  "Optimization  Set 
Up"  sheet  of  the  NDARC  spreadsheet,  which  is  labeled  below  in  Figure  7.  This  interface  provides  the  user 
with  the  ability  to  change  which  variables  will  be  design  variables  (to  be  varied  during  the  optimization) 
versus  constant  parameters,  set  the  values  of  each  variable,  set  a  filename  to  save  the  optimization  runs 
to,  and  a  "Run  Optimization"  button  that  runs  the  entire  optimization  process  based  on  the  information 
in  the  current  spreadsheet. 


|  Filename  used  to  save  results  to 

- -—i - 

Run  Optimization 

Reset  all  variables 

to  Default  Values 

with  current  set  up 

i 

lEntf  8w»*i  FBmime  Hf»  1  moahc  aawuwi  | 


Figure  7  "Optimization  Set  Up"  sheet  of  NDARC  spreadsheet  used  to  set  up  optimization  problem 


As  noted  in  Figure  7,  the  current  design  variables  being  considered  for  the  optimization  problem  have  a 
green  shaded  background  in  the  spreadsheet,  while  all  constant  parameters  have  white  backgrounds.  To 
change  a  variable  between  a  design  variable  and  a  constant  parameter,  the  user  simply  has  to  double  click 
on  the  variable  name  itself,  as  clearly  shown  in  Figure  8. 
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In  addition,  the  values  that  the  user  must  set  for  each  variable  are  dependent  on  the  type  of  variable. 
Constant  parameters  require  only  a  default  value  to  be  set,  which  is  simply  the  constant  value  they  will 
be  held  at  during  the  optimization  process.  For  design  variables,  the  NSGA-II  algorithm  requires  that  three 
values  be  provided:  a  lower  bound,  upper  bound,  and  a  resolution.  These  three  values  are  required 
because  the  NSGA-II  is  a  genetic  algorithm,  so  all  continuous  variables  must  be  discretized  to  some  finite 
resolution,  with  each  variable  bounded  on  both  sides.  To  make  it  clear  to  the  user  what  values  must  be 
provided,  only  the  necessary  inputs  for  each  variable  are  visible.  This  is  clearly  shown  in  Figure  7,  where 
the  design  variables  have  values  visible  in  the  “Lower  Bound",  "Upper  Bound",  and  "Resolution"  columns, 
while  the  constant  parameters  only  have  values  visible  in  the  "Default  Value"  column. 


Rotor  Induce 

d  Power  Variables 

- —  -  - 

MODEL  me 

imduced  velocity  factors  (ratio  to  momentum 

teorv  indue 

icity) 

profile  power 

TECH  drag 

Ki  hover 

Reference  Reynolds  numcer  (0  for  no  correction) 

Re  ret 

Ki_cHmB 

Basic  model  (1  array.  2  equation) 

MODEL  Basic 

Ki  Drop 

numoer  of  points  (maximum  25) 

- 1 - 1 - 1 - 24 

Variation  with  Iniu-vl 

Equalion 

CTstciKi  n  variation 

CTs  Hind 

CTfs  for  minimum  profile  Crag 

CTs  Dmin 

0  07 

coemoentforKi  n 

Khi 

coefficient  in  drao  vs  CT/s  function  (constant  (or  hover/ec 

-V. 

00  hei 

0  009 

Wi2 

coefficient  in  drag  vs  CT/s  function  (constant  tor  axial) 

00  prop 

0  009 

eiDonentforKi  n 

Xh2 

coefficient  in  drag  vs  CT/s  function  (linear  hcrver/edgewis 

dl  hel 

CTs  Pind 

coefficient  in  drag  vs  CT/s  function  ilinear  for  axial) 

01  prop 

i.p1 

coefficient  in  dr3g  vs  CT/s  function  (quadratic  (or  hover/e 

...... 

02.  hel 

0  5 

coefficient  for  Ki  p 

*P2 

coefficient  in  drag  vs  CT/s  function  (quadratic  for  anal 

02 prop 

eiQOoem  iqi  Ki p 

XP2 

variation  with  shan  angle  coefficient  tor  cop 

dprop 

Venation  with  Shaft  Angle 

variation  with  shall  angle  exponent  lor  cdp 

Xprop 

coefficient  for  Ki  p 

kpa 

1 

CTs  sep 

•■ponenltoiKi  j 

xpa 

1 

dsep 

Variation  with  lift  Offset 

Xsep 

ko2 

between  design  variable 

kktneffirienl 

df2 

M  3x131 

d  constant  parameter 

tv  expWfi  l 

Xt 

xaual 

an 

MODEL  SUII 

coemo enttor k^ mCz,  wear i - 

P 

constant  in  stall  drag  increment 

1  f: 

ka 2 

factor  in  stall  dtag  increment 

ostaili 

Ka3 

0 

Factor  in  stall  dfag  increment 

dstail2 

Xa 

45 

exponent  in  stall  drag  increment 

Xstalil 

Venation  with  Edgewise  Velocity 

exponent  in  stall  drag  increment 

mu  edge 

035 

Variation  with  Lift  Onset 

kel 

08 

coefficient  for  f( offset) 

doi 

ke2 

0 

Factor  for  f(oflset) 

do2 

8 

ke3 

1 

dsa 

xe 

45 

CompressiBiiity  model  (0  none.  1  drag  divergence.  2  sin 

lant 

MODEL  comf 

kea 

Similarity  Model 

Ki  mm 

1  085 

Factor 

rSim 

1 

Ki  max 

10 

Blade  Bp  tfuckness-to-chord  ratio 

thick  up 

oos 

Drag  Divergence  Model  (D=(Mat  Mdd).  0cd-dVD*d2-D 

coefficient  in  drag  Increment 

0  056 

exponent  in  drag  increment 

Drag  Divergence  Mach  number  (Mdd  -  MddO  Mddcrc 

Mdd  at  zero  lilt 

1  1  1  1  0  88 

|  den, at.ve  with  nn - 

-  lUdgd  1_ 1_ 1_ 1 ° 21 

Figure  8  Demonstrating  how  to  change  variable  type  between  design  variable  and  constant  parameter 


A  limitation  of  this  process  is  that  a  value  must  be  provided  FOR  EVERY  column  for  the  specified  variable, 
as  the  optimization  code  is  reading  in  these  values  and  has  no  logic  embedded  within  it  to  assign  values 
to  variables  if  they  are  missing  from  the  spreadsheet  table.  That  is,  if  a  variable  is  a  design  variable,  then 
the  user  must  input  a  value  for  the  "Upper  Bound",  "Lower  Bound",  and  "Resolution".  The  "Default  Value" 
is  hidden  from  the  user  for  the  design  variables  as  it  is  not  required  for  the  optimization  algorithm,  but 
the  current  "Default  Value"  does  not  need  to  be  deleted;  it  can  be  left  as  is.  Likewise,  for  a  constant 
parameter  the  "Default  Value"  column  must  be  set,  while  the  values  for  the  "Upper  Bound",  "Lower 
Bound",  and  "Resolution"  can  be  left  as  is,  but  will  be  hidden  from  the  user.  Because  of  this,  checks  have 
been  built  into  the  VBA  script  to  ensure  that  the  proper  values  have  been  assigned.  Upon  clicking  the  "Run 

Optimization"  button,  the  VBA  code  will  check  all  of  the  inputs,  and  provide  alert  messages  if  any  input 
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values  are  missing.  A  few  examples  of  this  are  shown  below  in  Figure  9.  The  alerts  will  tell  the  user  what 
variable  to  look  at,  what  table  the  variable  is  in  (either  induced  or  profile  power),  and  it  will  select  the  cell 
that  needs  to  be  changed. 


l-aj 


Default  Value  Missing 


Please  enter  value  for  Default  Value  for  Ki_prop  in  the  Induced  Power 
'  table 


LowerBound  Value  Missing 


Figure  9  Possible  error  messages  that  occur  when  NDARC  variables  are  not  set  correctly 


2.1.2. 5  Running  NSGA-11  Optimization 

To  determine  the  Pareto  Frontier,  the  Non-Dominated  Sorting  Genetic  Algorithm  II  (NSGA-II)  was 
implemented.  The  NSGA-II  algorithm  is  a  multi-objective  evolutionary  algorithm  that  optimizes  a 
population  of  points  to  approximate  the  Pareto  frontier  of  the  design  space.  The  result  of  the  NSGA-II 
algorithm  is  thus  an  estimation  of  the  Pareto  Frontier  of  the  design  space,  which  then  allows  for  the  use 
of  the  multi-objective  decision  making  technique  to  select  the  best  compromise  design.  The  NSGA-II 
algorithm  is  implemented  externally  to  the  NDARC  spreadsheet  in  Python,  which  has  been  compiled  into 
the  "RunNDARC_Optimization.exe"  executable.  The  executable  file  reads  the  NDARC  spreadsheet  to 
obtain  the  necessary  information  to  perform  the  optimization.  For  more  information  on  the  NSGA-II 
algorithm,  refer  to  the  paper  by  Deb  et.al  [3], 


2. 1.2. 6  Pareto  Frontier  of  Calibration  Data  Set 

Once  the  NSGA-II  algorithm  has  determined  the  Pareto  Frontier  of  the  design  space,  the  information  must 
be  loaded  back  into  the  NDARC  spreadsheet  so  that  it  can  be  accessed  to  make  an  informed  decision  on 
the  best  set  of  NDARC  variables.  This  is  automatically  done  within  the  NDARC  Spreadsheet,  which  loads 
the  calculated  errors  and  associated  configuration  settings  into  a  table  on  a  sheet  labeled  "Pareto  Frontier 
Configurations".  Because  there  is  no  need  for  the  user  to  interact  with  this  sheet,  and  it  in  fact  should  not 
be  changed  by  the  user  at  all,  this  sheet  should  in  general  be  hidden.  For  reference,  a  sample  table  is 
shown  below  in  Figure  10. 
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Figure  10  Sample  of  the  Pareto  Frontier  data  on  the  "Pareto  Frontier  Configurations"  sheet 


2.1.2. 7  Multi-Objective  Decision  Making  Technique  (TOPSIS) 

The  multi-objective  decision  making  (MADM)  technique  implemented  in  the  NDARC  spreadsheet  is  the 
Technique  for  Order  of  Preference  by  Similarity  to  Ideal  Solution  (TOPSIS).  Details  of  TOPSIS  can  be  found 
by  a  simple  web  search  for  "TOPSIS".  Essentially,  the  TOPSIS  technique  requires  that  the  user  place 
weightings  on  the  importance  of  the  different  requirements  or  design  objectives,  and  then  calculates  the 
best  (positive  ideal)  and  worst  (negative  ideal)  possible  solution  based  on  these  weightings.  The  different 
designs  are  then  ranked  based  on  their  distances  from  the  positive  and  negative  ideal  solutions.  TOPSIS 
simply  provides  a  way  to  rank  the  designs  on  the  Pareto  Frontier  to  determine  the  best  "compromise" 
design  based  on  those  weightings  of  the  design  objectives.  It  should  be  noted  that  this  is  purely  a  tool  to 
aid  the  user  in  making  a  decision,  and  that  other  MADM  techniques  existthat  will  provide  different  results, 
which  is  purely  due  to  difference  in  implementation. 

TOPSIS  is  implemented  on  the  NDARC  spreadsheet  in  two  separate  sheets.  The  user  interface  is  on  the 
"TOPSIS"  sheet,  while  all  calculations  required  for  the  TOPSIS  are  all  performed  on  a  sheet  called  "TOPSIS 
Calculations".  Like  the  "Pareto  Frontier  Configurations"  sheet,  the  user  should  not  change  anything  on  the 
"TOPSIS  Calculations"  sheet,  and  for  this  reason  the  sheet  is  hidden  from  the  user. 

The  user  interface  on  the  "TOPSIS"  sheet  is  shown  in  Figure  11.  The  interface  has  two  slider  bars  that  allow 
the  user  to  weight  the  importance  of  minimizing  the  error  in  the  forward  flight  and  hover  conditions, 
respectively.  The  weightings  are  normalized  such  that  they  always  sum  to  one;  thus,  an  equal  weighting 
implies  that  reducing  the  error  in  both  flight  conditions  is  of  equal  importance.  The  table  shown  displays 
the  Top  10  ranked  configurations  or  "Cases"  from  the  current  TOPSIS  analysis,  along  with  the  magnitude 
of  the  calculated  error  and  the  NDARC  design  variables  that  each  case  represents.  The  "Case"  number 
simply  represents  the  case  number  assigned  to  each  configuration  in  the  table  on  the  "Pareto  Frontier 
Configurations"  sheet.  Any  time  that  a  slider  bar  is  moved,  the  NDARC  spreadsheet  will  automatically  be 
updated  to  reflect  the  design  that  is  ranked  number  1.  This  is  reflected  in  the  graphs  that  represent  the 
induced  power  and  profile  power  plots. 
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If  the  user  would  like  to  see  the  results  of  a  case  that  is  not  ranked  1,  they  can  enter  that  case  number  in 
the  "Case  Number"  box  and  click  the  "Update  Configuration  Case"  button.  This  will  load  the  specified  case 
number  into  the  NDARC  spreadsheet,  which  again  will  be  reflected  by  the  induced  power  and  profile 
power  plots. 
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Figure  11  Sample  TOPSIS  user  interface 
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2.2  Applicable  Rotorcraft  T echnologies 


2.2.1  Introduction  to  Technology  Identification 

In  order  to  better  capture  technology  impacts  within  the  CATE  environment,  an  extensive  rotorcraft 
technology  literature  search  was  performed.  The  focus  of  this  literature  search  was  on  single  main  rotors 
(i.e.  UH-60).  During  the  literature  search,  different  technologies  were  identified,  along  with  their  impacts 
on  the  various  components  of  the  rotorcraft  (i.e.  physical/functional).  This  led  to  a  rotorcraft  technology 
taxonomy  where  the  physical  and  functional  decompositions  of  technologies  were  categorized.  Finally, 
research  was  conducted  to  determine  how  to  best  represent  these  technologies  effectively  and  efficiently 
within  the  CATE  environment. 

2.2.2  Process 

This  process,  outlined  in  Figure  12  below,  begins  with  a  literature  search  to  discover  the  emerging 
rotorcraft  technologies.  This  literature  search  encompassed  many  reports  and  papers  relating  to 
rotorcraft  technologies.  The  next  step  was  to  determine  which  areas  of  the  rotorcraft  system  were 
affected  by  each  technology,  which  was  accomplished  in  two  components  as  described  below. 


•  ASSP  Goals 

•  Technology  Taxonomy 

•  ASSP  to  NDARCTech 
Factors  Spreadsheet 

Bl  •  Subject  Matter  Experts 

I  •  Higher  Fidelity  Analysis  Tools 

Figure  12  Technology  Identification  Process 

The  first  component  of  the  vehicle  impact  assessment  step  was  to  determine  an  overall  breakdown  of 
where  each  system  of  technologies  is  located  on  the  vehicle,  which  resulted  in  the  construction  of  a 
rotorcraft  taxonomy.  This  taxonomy  includes  major  rotorcraft  systems  such  as:  the  rotor,  engine, 
transmission,  etc.  These  groups  of  systems  were  further  broken  down  based  on  the  function  of  the 
individual  technologies.  For  example,  rotor-related  technologies  were  broken  down  into  technologies 
related  to  active  rotor  systems,  rotor  planform  alterations,  and  retreating  stall  delay.  This  grouped 
technology  list,  including  system  breakdowns,  can  be  seen  in  the  technology  taxonomy  in  Figure  13. 
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Applicable 

Rotorcraft 

Technologies 


Retreating  Blade  IR  Signature  Component  Component 

Stall  Delav  HI  Reduction  Technologies  Technologies 


Plasma  Flow 
Control 


Infrared 

Suppression 

Systems 


Ceramic  Matrix 
Composites 


Improved  Turbine 
Engine  (ITEP) 


Anhedral/Dihedral 


Figure  13  Identified  Technology  Taxonomy 


The  second  component  of  this  step  was  to  break  down  specifically  where  each  technology  will  impact  the 
vehicle.  Because  this  is  a  more  complicated  component,  it  was  important  to  look  into  how  the  current 
rotorcraft  world  breaks  down  the  rotorcraft  from  a  technology  point  of  view.  This  search  resulted  in  the 
discovery  of  the  Aviation  Science  &  Technology  Strategic  Plan  (ASSP).  This  plan  includes  future  objectives 
for  various  rotorcraft  systems  and  is  broken  down  into  various  focus  areas.  The  focus  areas  applicable  to 
the  technology  research  are  shown  in  Figure  14.  Each  ASSP  focus  area  is  further  broken  down  into  more 
specific  components,  as  shown  in  Figure  15. 
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Continuous  Trail  ng  Edge  Rap  (CTEF) 


Active  Rotor  Control  Systems 
Rotor  Planform  Alterations 
Retreating  Blade  Stall  Delay 
Control  Systems 


Wide  Chord  &i»de  <WCB| 


Health  Monitoring 

§7- 

Health  &  Usage  Mon  boring  System  (HUMS) 


Fiy-by-Wire/L-ght 


Hybrid  Gears 

Engine  Component  Technologies 
Transmission  Component  Technologies 


IR  Signature  Reduction 


Infrared  Suppression 
Systems 

Figure  14  ASSP  Focus  &  Technology  Areas 


C:%  a 

Improved  Turbine  Engine 
Program  (fTEP| 


Platforms 

•  Structures 

•  Aeromechanics  /  Rotors 

•  Vehicle  Management  &  Control 

•  Subsystems 


Mission  Systems 

•  Engagement  and  Effects 

•  Survivability 

•  Teaming,  Autonomy  & 
Information  Management 

•  Human  System 
Interface 

•  Avionics/ 

Networking 


Major  Program  Areas 


Basic  Research 


Figure  15  ASSP  Focus  Area  Breakdown  [4] 


This  focus  area  breakdown  is  then  further  broken  down  into  specific  technology  objectives.  This  helped 
to  determine  what  kind  of  impacts  a  given  technology  may  have.  For  example,  the  Aeromechanics 
subgroup  of  the  Platforms  group  is  broken  down  as  shown  in  Figure  16. 
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Reduction  in  vibration 


Reduction  in  Adverse  Aero  Forces 


Reduction  in  Gust  Response  & 
Loading 


Increase  in  Hover  Efficiency 
(Helicopter/Tilt  Rotor) 


Increase  in  Rotor  Forward  Flight 
Efficiency 


Increase  in  Max  Blade  Loading 


Decrease  in  Acoustic  Detection 
Range 


Reduction  in  Rotor  Lag  Damping  Cost 


Aeromechanics  Modeling,  Accuracy 


Increase  in  Control  Effectiveness 


Figure  16  Aeromechanics  Breakdown 


Due  to  the  fact  that  CATE  uses  surrogates  powered  by  NDARC,  each  ASSP  technology  objective  was  then 
mapped  to  specific  NDARC  parameters.  This  helped  to  determine  which  NDARC  parameters  (or  tech 
factors)  were  affected  by  each  technology  based  on  the  technology's  expected  impact.  A  snapshot  of  such 
mapping  is  found  in  Figure  17.  The  blue  fill  in  Figure  17  indicates  that  the  tech  factor  above  related  to  the 
ASSP  technology  objective  to  the  left. 


NDARC 

Tech 

Factors 


The  final  step  in  this  process  was  to  determine  the  specific  NDARC  parameter  values  associated  with  each 
technology.  This  process  is  different  for  each  technology  but  has  some  inherent  similarities.  Using  the 
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information  found  in  the  literature  search  it  is  possible  to  determine  these  parameters  in  a  number  of 
ways.  The  first  is  to  use  direct  percentage  decreases  (or  increases)  to  specific  parameters  found  in  the 
literature.  For  example,  if  an  engine  technology  is  expected  to  reduce  SFC  by  10%,  the  SFC  NDARC 
parameter  is  multiplied  by  0.90.  Another  possible  method  is  to  use  expected  performance  improvements 
of  a  given  technology  and  reverse  engineer  the  associated  tech  factors.  For  example,  if  an  engine  is 
expected  to  have  a  given  power  available  at  various  altitudes  and  velocities,  these  "sweeps"  can  be 
created  in  NDARC  and  various  tech  factors  can  be  varied  in  order  to  match  the  power  available  within  an 
acceptable  error  threshold.  The  final  method  is  to  use  the  knowledge  of  subject  matter  experts  (SME's). 
Their  knowledge  about  where  a  technology  is  expected  to  be  at  in  the  future  will  help  determine  what 
modeling  parameters  need  to  be  changed  and  by  what  amount. 

For  each  technology  listed  in  Figure  13,  the  associated  technology  objectives  and  resulting  NDARC  tech 
factors  were  identified.  For  some  of  the  technologies,  actual  NDARC  tech  factor  values  were  determined. 
The  technologies  are  detailed  in  the  following  section. 

2.2.3  Identified  Technologies 

The  full  list  of  applicable  rotorcraft  technologies  identified  is  shown  in  Figure  13.  As  previously  stated,  it 
is  important  to  note  that  this  list  is  not  all-inclusive.  There  are  many  more  rotorcraft  technologies  being 
considered,  but  this  method  can  be  applied  to  more  in  the  future  and  will  allow  researchers  to  determine 
where  the  technologies  fit  into  the  aforementioned  taxonomy. 

2.2.3.1  Rotor  Technologies 

Continuous  Trailing  Edge  Flap  (CTEF) 

Unlike  a  conventional  flap,  the  Continuous  Trailing  Edge  Flap  (CTEF),  as  the  name  implies,  does  not  have 
a  break  in  the  trailing  edge  of  the  wing.  Developed  by  the  Army  Research  Lab  (ARL),  the  CTEF  utilizes  an 
optimized  biomorph  designed  with  Macro-Fiber  Composite  (MFC)  actuators  to  change  the  camber  of  the 
airfoil  section  in  order  to  provide  primary  flight  control  for  a  rotorcraft  (both  collective  and  cyclical  pitch 
controls)  [5],  According  to  ARL  in  regards  to  the  CTEF,  "more  efficient  aerodynamic  excitation  combined 
with  a  simplified  structural  design  will  reduce  vibration  and  permit  in-flight  blade  tracking,  thereby 
reducing  maintenance  costs  for  Army  rotorcraft"  [6],  A  depiction  of  the  CTEF  can  be  seen  in  Figure  18 
below. 
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Figure  18  Cross-sectional  View  of  the  CTEF 


Given  the  CTEF  is  a  rotor-related  technology,  most  of  its  impacts  are  within  the  Aeromechanics  group  of 
the  ASSP  goals.  The  following  list  includes  the  technology  objectives  that  may  be  affected  by  including  the 
CTEF  on  the  vehicle: 

•  Reduction  in  Vibration 

•  Reduction  in  Adverse  Aero  Forces 

•  Increase  in  Flover  Efficiency 

•  Decrease  in  Acoustic  Detection  Range 

Due  to  the  complexity  of  the  CTEF,  it  is  expected  that  there  may  be  an  increase  in  manufacturing  cost, 
which  is  contrary  to  "Reduction  in  Manufacturing  Cost"  technology  objective  within  the  structure  group. 
Using  the  ASSP  Technology  Objective  to  NDARC  tech  factor  mapping  spreadsheet,  along  with  some  good 
engineering  judgment,  the  NDARC  parameter  set  shown  in  Table  2  was  selected  to  be  used  to  model  the 
CTEF  within  NDARC.  Utilizing  the  results  of  a  questionnaire  given  to  a  subject  matter  expert  done  in  a 
previous  year  of  the  project,  NDARC  parameters  were  varied  to  match  the  expected  SME  projections.  The 
NDARC  parameters  affected  by  the  CTEF  and  their  resulting  percent  changes  are  shown  in  Table  2.  It  is 
important  to  note  that  because  the  reduction  in  vibration  cannot  be  directly  modeled  in  NDARC,  it  is 
assumed  that  the  reduction  in  vibration  due  to  the  CTEF  can  be  related  to  a  reduction  in  maintenance 
costs,  as  less  rotor  vibration  would  lead  to  less  wear  and  tear  on  the  rotors. 


Table  2  CTEF  NDARC  Parameters 


Baseline  NDARC  Factor 

Percent  Change  from  Baseline 

Flover  Induced  Drag 

-1% 

Fuselage  Body  Weight 

3.3% 

Maintenance  Cost 

-24% 
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Wide  Chord  Blade 

Another,  more  immediate,  rotor  technology  is  the  Wide  Chord  Blade  (WCB)  (shown  in  Figure  19).  The 
WCB  offers  increased  lift  due  to  a  16%  wider  chord  blade  compared  to  common  rotors  [7],  It  is  also 
constructed  using  advanced  composites,  rather  than  traditional  rotor  materials.  Currently  on  the  UH-60M 
rotor  system,  the  WCB  generates  an  additional  470  pounds  of  lift  and  its  advanced  design  improves 
maneuverability  and  air  speed  [8],  Its  all-composite  spar  also  reduces  maintenance  man  hours. 


Figure  19  Wide  Chord  Blade 

A  literature  search  for  the  WCB  system  resulted  in  two  useful  sources  of  information  for  assessing  the 
new  rotor.  Yeo  et  al.  modeled  the  WCB  system  in  a  high  fidelity  code  and  found  that  the  increase  in  solidity 
was  the  main  performance  driver  because  of  de-loading  the  blades  [9],  Thus  it  was  assumed  that  the 
details  of  the  aerodynamics  would  be  captured  by  changing  blade  loading,  which  is  a  parameter  on  the 
TIM.  It  was  found  that  the  10%  increase  in  solidity  reported  for  the  WCB  by  Yeo  et  al  results  in  a  9.1% 
decrease  in  blade  loading  [9], 

Weight  changes  were  based  on  results  from  Nixon's  paper,  which  focused  on  modeling  the  structure  of  a 
composite  rotor  blade  and  using  optimization  to  find  minimum  weight  designs.  His  paper  used  the  UH- 
60A  as  a  validation  case.  Nixon's  results  for  estimating  blade  weight  changes  due  to  composite  designs 
were  based  on  the  aerodynamics  of  the  UH-60A.  Nixon\concluded  that  a  single-spar  composite  design 
would  result  in  a  21.3%  weight  reduction  and  a  multi-spar  composite  design,  which  would  be  inherently 
more  ballistically  tolerant,  would  result  in  a  12.1%  weight  reduction  relative  to  the  metallic  design  used 
for  the  UH-60A  [10],  It  is  assumed  that  this  weight  reduction  due  to  composite  materials  is  applicable  to 
the  WCB  because  it  uses  an  all  composite  blade. 
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Given  the  year  of  Nixon's  paper  (1987),  these  weight  estimates  have  a  good  deal  of  uncertainty.  With  the 
uncertainty  in  the  weight  reduction  due  to  composite  materials,  the  wide  chord  blade  technology  offers 
a  good  case  for  using  distributions  bounded  by  no  weight  change  and  a  21.3%  weight  decrease.  However, 
for  cases  where  distributions  cannot  be  used,  and  because  we  do  not  know  which  structural  design  was 
used,  the  conservative  prediction  of  12.1%  was  selected.  There  was  no  specific  information  found  on  how 
the  control  weight  would  change,  so  no  assumptions  were  made  as  to  potential  technology  impacts  for 
these.  Finally,  impacts  for  other  technology  factors,  such  as  survivability  or  maintainability,  were  not 
researched  given  the  performance  focus  of  the  use  case.  However,  future  work  for  demonstrating  the 
maintenance  discrete  event  simulator  can  use  the  WCB  as  an  example  technology. 

From  this  literature  search,  it  was  determined  that  the  following  ASSP  technology  objectives  were  e 
affected  by  the  WCB  system. 

•  Increase  in  Max  Blade  Loading 

•  Increase  in  Control  Effectiveness 

•  Reduction  in  Structural  Maintenance  Labor 

•  Increase  in  Power  to  Weight 

Utilizing  the  knowledge  gained  from  the  literature  search,  as  well  as  the  ASSP  technology  objectives  under 
consideration,  Table  3  summarizes  the  changes  to  baseline  NDARC  parameters  for  the  WCB. 


Table  3  Wide  Chord  Blade  NDARC  Parameters 


Baseline  NDARC  Factor 

Percent  Change  from  Baseline 

Blade  Loading 

-9.1% 

Blade  Weight 

-12.1% 

Plasma  Flow  Control 

According  to  the  Army  Research  Laboratory  (ARL),  "plasma  based  flow  control  is  a  potential  active  rotor 
technology  that  could  lead  to  rotorcraft  performance  enhancement  without  increasing  the  rotor  size." 
This  would  lead  to  an  increased  payload  capacity,  higher  achievable  speeds,  and  increased  range 
capabilities  [11].  This  could  solve  the  dynamic  stall  problem  that  rotor  blades  can  be  prone  to 
experiencing.  Plasma  based  flow  control  delays  the  onset  of  flow  separation  (or  stall).  Figure  20  below 
illustrated  flow  visualizations  for  an  uncontrolled  airfoil  vs.  a  plasma  flow  controlled  airfoil. 
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Uncontrolled  Controlled 


Figure  20  Uncontrolled  vs.  Controlled  Plasma  Flow  Visualization 


Most  of  the  impacts  of  plasma  flow  control  are  in  the  Aeromechanics  technology  objective  group.  The 
following  technology  objectives  have  been  identified  to  be  potentially  impacted  by  this  technology: 

•  Increase  in  Rotor  Forward  Flight  Efficiency 

•  Increase  Max  Blade  Loading 

These  technology  objective  have  all  been  identified  because  they  will  benefit  from  the  reduced  flow 
separation  (i.e.  stall)  tendency  that  occurs  at  high  forward  flight  velocities.  Without  aerodynamic  losses 
due  to  stall,  the  rotor  will  have  an  increased  forward  flight  efficiency  and  be  able  to  sustain  a  higher 
loading.  It  also  may  provide  the  possibility  for  smaller  rotor  blades. 

Given  these  technology  objectives,  the  following  NDARC  parameters  have  been  identified  as  possibly 
being  impacted  by  the  plasma  flow  control: 


Table  4  NDARC  Parameters  Affect  by  Plasma  Flow  Control 


NDARC  Parameter 

Description 

Ki prop 

Axial  Cruise  Induced  Velocity  Factor 

CWs 

Blade  Loading 

2.2.3.2  Engine  Technologies 
Improved  Turbine  Engine  Program  (ITEP) 

As  the  name  implies,  the  Improved  Turbine  Engine  Program  is  the  US  Army's  initiative  to  develop  a  new 
turbine  engine  that  weighs  the  same  as  the  current  UH-60L  engine  (the  GE  T700-GE-401C  at  456  pounds) 
but  produces  30%  more  shaft  horsepower  (increasing  lift  capacity  by  27%),  all  while  consuming  25%  less 
fuel  [12]. 
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After  conducting  a  literature  search,  the  following  ASSP  technology  objectives  were  identified  that  may 
be  impacted  by  the  ITEP  engine. 

•  Lower  SFC 

•  Increase  in  Power  to  Weight 

The  engine  is  also  expected  to  have  a  higher  unit  cost  which  would  increase  engine  procurement  cost  [12]. 
This  is  contrary  to  the  "Reduction  in  Engine  Procurement  Costs"  ASSP  technology  objective. 

Utilizing  the  information  obtained  in  the  literature  search,  it  was  possible  to  determine  certain  NDARC 
parameters  that  needed  to  be  changed.  As  mentioned  above,  the  ITEP  engine  is  expected  to  have  a  power 
rating  of  3,000  hp  IRP  (30%  more  than  the  GE  T700-GE-401C)  and  a  weight  of  456  pounds.  The  power 
improvement  can  be  modeled  directly  in  NDARC  by  increasing  the  Peng  parameter  to  3000.  This 
automatically  increases  the  engine  weight  when  an  NDARC  sizing  run  is  initiated.  In  order  to  counter  this 
weight  increase,  the  TECH_eng  parameter  was  varied  utilizing  an  optimization  routine  as  a  numerical 
solver  to  match  the  NDARC  output  engine  weight  to  the  expected  456  pounds.  The  results  of  this 
demonstration  and  the  NDARC  parameters  that  were  changed  are  found  in  Table  2. 


Table  5  ITEP  Engine  Demonstration 


NDARC  Input  Parameter 

Parameter  Value 

SLS  Engine  Power 

3000 

Engine  Weight  Parameter 

1.773 

NDARC  Output  Parameter 

Parameter  Result 

Engine  Weight 

456.01 

Ceramic  Matrix  Composite  (CMC)  Components 

As  improving  engine  efficiency  has  always  been  a  goal  within  the  aerospace  community,  ways  to  improve 
the  turbine  engine  (used  on  rotorcraft  systems)  have  been  looked  into.  One  such  way  is  to  utilize  Ceramic 
Matrix  Composites  (CMC)  for  components  within  the  turbine  engine.  These  CMC  components  offer 
benefits  of  higher  temperature  capability  and  less  cooling  requirements.  This  leads  to  improved  efficiency 
and  reduced  emissions  [13].  Figure  21  illustrates  a  concept  to  utilize  CMC's  to  develop  turbine  vanes. 
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Figure  21  Ceramic  matrix  composite  turbine  vane  concept 


The  following  ASSP  technology  objective  would  be  impacted  by  the  use  of  Ceramic  Matrix  Composites 
within  the  turbine  engine  (all  within  the  Engines  group)  [14]: 

•  Reduction  in  Engine  Operating  Cost 

•  Lower  SFC 

•  Increase  in  Power-to-Weight 

These  would  all  have  positive  direction  of  improvement  by  the  CMC  components.  The  NDARC  parameters 
that  would  potentially  be  impacted  by  the  CMC  components  are  listed  in  Table  6.  In  addition,  utilizing  a 
higher  fidelity  engine  design/analysis  tool  (i.e.  NPSS)  would  allow  a  technologist  to  obtain  a  more  accurate 
representation  of  the  CMC  components'  impact  on  the  engine. 


Table  6  NDARC  Parameters  Impacted  by  CMC  Components 


NDARC  Parameter 

Description 

Tech cost maint 

Maintenance  Cost  Technology  Factor 

sfcOC tech 

Specific  Fuel  Consumption  at  MCP  Technology  Factor 

Tech_eng 

Engine  Weight  Technology  Factor 

Hover  Infrared  Suppression  System  (HIRSS) 

One  of  the  most  prominent  sources  of  infrared  detection  on  a  rotorcraft  is  its  hot  engine  exhaust  gases, 
as  can  be  seen  in  Figure  22.  Reducing  the  infrared  (IR)  signature  of  rotorcraft,  specifically  those  used  in 
military  applications,  is  highly  desirable.  Currently  equipped  on  the  UH-60M,  the  Hover  Infrared 
Suppression  System  (HIRSS)  provides  shielding  of  hot  engine  exhaust  gases  in  order  to  reduce  the  aircraft's 
infrared  signature.  This  reduces  the  aircraft's  susceptibility  to  be  detected. 
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Figure  22  Rotorcraft  infrared  (IR)  signature 


The  following  is  a  list  of  the  ASSP  technology  objectives  that  may  be  affected  by  the  HIRSS.  The  list  includes 
a  combination  of  technology  objectives  from  the  Survivability,  Aeromechanics,  and  Engines  groups. 

•  Survivability 

o  Reduced  Visual/Infrared/Electro-Optical  Signature 

•  Aeromechanics 

o  Increase  in  Hover  Efficiency 
o  Increase  in  Rotor  Forward  Flight  Efficiency 

•  Engines 

o  Increase  in  Power-to-Weight 

Of  these  technology  objectives,  only  the  one  in  the  Survivability  group  is  actually  improved.  The  goal  of 
the  HIRSS  is  to  reduce  the  infrared  signature.  The  others  actually  see  a  degradation  in  performance.  With 
the  HIRSS  turned  on,  the  engine  efficiency  is  deceased  due  to  minimal  power  losses  [15].  The  engine 
weight  may  also  increase  due  to  the  addition  of  such  system,  reducing  the  power-to-weight  of  the  engine. 

Utilizing  the  ASSP  Technology  Objective  to  NDARCTech  Factor  mapping,  the  following  NDARC  parameters 
(Table  7)  have  been  identified  as  potentially  being  impacted  by  the  HIRSS.  It  is  important  to  note  that  the 
Survivability  aspects  of  the  HIRSS  cannot  be  modeled  within  NDARC  as  it  is  simply  a  rotorcraft  design  tool. 
Such  parameters  can  be  modeled  in  an  Operations  Model,  such  as  the  Discrete  Event  Simulator  being 
developed  as  part  of  the  overall  CATE  efforts. 
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Table  7  NDARC  parameters  impacted  by  HIRSS 


NDARC  Parameter 

Description 

TECH drag 

Profile  Power  Technology  Factor 

TECH_eng 

Engine  Weight  Technology  Factor 

2. 2. 3. 3  Transmission 

Hybrid  Gears 

Though  the  use  of  composites  in  drive  train  systems  is  limited,  hybrid  gears  are  way  to  combine  light¬ 
weight,  high-strength  composites  with  traditional  metallic  materials  in  order  to  provide  a  very  high  power 
to  weight  ratio  [16].  A  20%  decrease  in  weight  as  well  as  a  reduction  in  noise  and  vibration  can  possibly 
be  achieved  [16].  A  hybrid  gear  is  illustrated  in  Figure  23  below. 


Figure  23  Hybrid  Gear 

Most  of  the  impacts  of  the  hybrid  gears  are  located  within  the  drive  system  of  the  rotorcraft.  The  following 
list  includes  the  technology  objectives  that  may  be  affected  by  including  the  CTEF  on  the  vehicle. 

•  Lower  SFC 

•  Increase  in  Power-to-Weight 

•  Reduction  in  Acquisition  Cost 

•  Reduction  in  Drive  System  Generated  Noise 

Utilizing  the  SME  questionnaire,  the  following  NDARC  parameters  are  affected  with  the  associated 
percent  change  from  baseline  (Table  8). 


Table  8  Hybrid  Gears  NDARC  Parameters 


Baseline  NDARC  Factor 

Percent  Change  from  Baseline 

Gear  Box  Weight 

-1.5% 

Specific  Fuel  Consumption  at  MCP 

-3% 

33 


2.2.3A  Other  Technologies 


Health  and  Usage  Management  Systems  (HUMS) 

In  an  effort  to  reduce  system  failures  on  rotorcraft,  systems  are  being  developed  to  anticipate  critical 
maintenance  needs.  Once  such  system  is  the  Health  and  Usage  Management  System  (HUMS),  which  is 
described  briefly  in  Figure  24. 


Figure  24  Health  and  Usage  Management  System  Schematic 


HUMS  is  a  fleet-wide  maintenance  management  system  provides  the  following  functions  [17]: 

•  Engine  Performance  Assessment 

•  Rotor  Track  and  Balance  (RTB) 

•  Absorber  Tuning 

•  Mechanical  Diagnostics 

•  Exceedance  Monitoring 

•  Usage  Monitoring 

•  Ground  Station  Processing 

With  these  functions,  the  goal  of  HUMS  in  to  reduce  fleet  operating  costs  and  improve  performance  by 
monitoring  the  usage  and  health  of  a  vehicle  [18].  The  following  ASSP  technology  objectives  have  been 
identified  as  potential  areas  of  improvement  with  the  addition  of  HUMS  to  the  aircraft: 

•  Sustainment 

o  Automatic  Detection/Diagnostics  of  Critical  Component  Failures 
o  Prognostics  of  Life/Maintenance  Actions  for  Critical  Components 

•  Subsystems 

o  Decrease  Maintenance  Man-Hours 
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Given  the  nature  of  the  HUMS  technology,  the  only  NDARC  tech  factor  that  is  affected  is 
TECH_cost_maint,  which  is  the  Maintenance  Cost  Technology  Factor.  The  other  factors  can  only  be 
considered  in  an  Operations  Model,  such  as  the  Discrete  Event  Simulator  being  developed  as  part  of  the 
overall  CATE  efforts. 

Fly-by-Wire/Fly-by-Light  Systems 

Fly-by-Wire  (FBW)  and  Fly-by-Light  (FBL)  systems  are  currently  used  on  many  fixed  wing  aircraft,  but  due 
to  the  complexity  of  the  control  system,  these  types  of  systems  are  not  yet  common  on  rotorcraft  systems. 
However,  efforts  are  currently  being  made  to  incorporate  these  systems  into  rotorcraft.  FBW  systems 
utilize  electrical  signals  to  move  actuators  to  deflect  control  surfaces  or  rotor  blades,  while  FBL  systems 
utilize  a  similar  structure  that  uses  fiber  optics  to  transmit  the  control  signals.  A  schematic  of  a  Fly-by- 
Wire  system  compared  to  a  traditional  mechanical  system  in  shown  in  Figure  25.  These  types  of  control 
systems  are  designed  to  improve  system  weight,  handling  qualities,  system  reliability,  and  maintenance 
time  due  to  a  reduction  in  moving  parts  that  ultimately  fail  less  frequently  than  conventional  mechanical 
systems  [19]. 


MECHANICAL  FLIGHT  CONTROLS 


ELECTRICAL  FLIGHT  CONTROLS  (FLY  by  WIRE) 


Utilizing  the  information  found  in  the  literature  search  of  these  systems,  the  following  ASSP  technology 
objectives  have  been  identified  as  potentially  impacted: 

•  Aeromechanics 

o  Increase  in  Control  Effectiveness 
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•  Structures 

o  Reduction  in  Structural  Wt/DGW 

•  Subsystems 

o  Decrease  Maintenance  Man-Hours 

These  systems  would  improve  all  of  these  objectives  as  aforementioned.  The  following  NDARC  parameters 
would  be  affected  based  on  the  ASSP  technology  objective  to  NDARC  tech  factor  mapping  described 
earlier: 


Table  9  NDARC  Parameters  Impacted  by  FBW/FBL  Systems 


NDARC  Parameter 

Description 

Tech_RWfc_b 

Boosted  Rotary  Wing  Flight  Control  Weight  Technology  Factor 

Tech_RWfc_mb 

Control  Boost  Mechanisms  Rotary  Wing  Flight  Control  Weight  Technology  Factor 

Tech_RWfc_nb 

Non-boosted  Rotary  Wing  Flight  Control  Weight 

TECH_RWhyd 

Rotary  Wing  Flight  Control  Hydraulics  Technology  Factor 

2.3  Assist  ARL  in  Integrating  Tools  into  OpenMDAO 

In  an  effort  to  support  ARL  in  the  integration  of  physics-based  models,  OpenMDAO  was  investigated  as  a 
possible  solution  to  solve  Multidisciplinary  Design  Analysis  and  Optimization  (MDAO)  problems.  The  work 
was  based  on  previous  analyses  made  by  ARL. 

OpenMDAO  is  a  Python-based  open  source  software  that  aims  to  integrate  multiple  disciplines  analysis 
together  and  find  optimal  solution  to  multi-disciplinary  problems.  It  is  developed  by  developed  at  NASA 
Glenn  Research  Center  which  provides  some  support  online[20].  The  most  recent  distributions  and 
archives  of  previous  versions  along  with  documentation  are  available  online  [21].  The  team  used 
OpenMDAO  v0.13  on  a  Windows  personal  machine.  It  was  noted  that  the  installation  requires  the 
following  software:  Python,  NumPy  and  SciPy  [22], 

Previous  ARL  work  [23]  [24]  includes  the  use  of  OpenMDAO  to  generate  NDARC  rotor  performance  maps 
from  published  data  and  to  integrate  RCAS,  a  comprehensive  rotor  analysis  code  with  NDARC.  The  NDARC 
wrapper  for  the  case  described  in  the  papers  was  provided  to  the  team  by  ARL.  Even  though  OpenMDAO 
showed  promise,  it  was  found  that  the  integration  of  the  codes  of  different  tools  (such  as  MATLAB  and 
importing  text  files)  was  not  straightforward  when  compared  to  similar  programs  designed  to  perform 
the  same  task,  such  as  Model  Center  and  ISIGHT. 
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The  work  focused  on  the  integration  of  the  UH-60  NDARC  files  in  OpenMDAO.  Running  the  UH-60  NDARC 
task  include  opening  the  description  files,  modifying  them  with  given  sizing  parameters,  running  the  sizing 
task  and  parsing  the  output.  The  description  files  include  the  mission  description,  engine  file  and  aircraft 
description.  Unfortunately,  the  engine  description  files  could  not  be  parsed  and  read  by  OpenMDAO  due 
to  numbering  convention  in  the  file.  The  problem  was  identified,  but  the  efforts  to  solve  it  were  put  on 
hold  while  the  other  tasks  of  the  current  report  were  being  performed. 
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2.4  Operation  and  Maintenance  Model 

The  primary  focus  of  the  FY16  work  was  to  refine  the  model  and  present  it  at  the  American  Helicopter 
Society  (AHS)  annual  forum.  Model  refinement  was  focused  on  improving  the  robustness  of  the  model 
and  preparing  it  for  integration  with  CATE.  While  working  through  the  model  some  critical  questions 
arose  with  regard  to  the  operations  portion  of  the  model.  Due  to  the  time  limitation  with  the  AHS 
paper  being  presented,  focus  was  placed  on  model  verification  with  the  questions  to  be  addressed 
after.  Following  the  presentation  at  the  forum,  discussions  with  the  Integrated  Product  Lifecycle 
Engineering  Laboratory  (IPLE)  on  campus  revealed  an  interesting  opportunity  to  improve  the  model. 

The  following  sub-sections  describe  the  overall  goal  of  the  operations  model,  the  Discrete  Event 
Simulation  (DES)  model  as  presented  at  the  AHS  forum,  its  limitations,  the  work  done  in  collaboration 
with  IPLE  to  improve  the  model,  and  future  work  for  FY17.  Furthermore,  research  into  the  addition  of 
a  combat  phase  to  the  operations  model  is  being  researched  and  initial  findings  are  presented  here. 

2.4.1  Introduction 

There  is  a  big  push  in  the  vertical  lift  community  to  develop  systems  that  are  reliable  and  maintainable, 
for  Department  of  Defense  (DoD)  acquisition.  The  DoD  Reliability,  Availability,  Maintainability  (RAM),  Cost 
Rationale  Report  Manual  (Ref.  25)  describes  'sustainment'  as  a  key  component  of  performance  and  claims 
including  sustainment  planning  upfront  enables  the  acquisition  and  requirements  communities  to  provide 
a  weapon  system  with  optimal  availability  and  reliability  to  the  warfighter  at  value.  'Sustainment'  is  made 
up  of  Availability  (Materiel  and  Operational),  Reliability,  and  Operations  and  Support  (O&S)  Cost.  This 
paper  will  discuss  the  use  of  an  integrated  discrete  event  simulation  model  to  estimate  RAM  for  rapid 
system  trade-off  analysis.  The  use  of  discrete  event  simulation  tool  is  essential  to  this  method  as  it  enables 
designers  to  evaluate  different  concepts  to  achieve  a  desired  Operational  Availability  (Ao)  and 
affordability. 

2.4.2  Model  as  Presented  at  AHS  Forum  72 

The  model  presented  at  the  72nd  American  Helicopter  Society  annual  forum  was  targeted  at  assessing 
technology  impacts  on  the  reliability,  availability,  and  maintainability  of  a  fleet  of  rotorcraft  through 
discrete  event  simulation  of  the  maintenance  and  operational  lifecycles  of  the  fleet.  Vehicles  were 
modeled  as  a  container  of  parts,  each  of  which  accrued  damage  through  normal  fatigue  during  flight. 
Incorporation  of  technology  factors  allowed  exploration  of  technology  effects  on  operational 
availability,  vehicle  loss  rate,  operations  and  support  costs,  and  maintenance  metrics.  Initial  model 
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verification  on  a  UH-60M  baseline  demonstrated  expected  trends  in  availability  as  well  as  the  ability 
to  model  technologies  which  impact  O&S  costs,  such  as  HUMS. 


2.4.2.1  Modeling  criteria  selection 

Affordability,  availability,  and  maintainability  were  selected  as  the  metrics  to  be  calculated  with 
the  initial  modeling  capability  described  below.  These  objectives  share  a  common  thread  of 
operations  and  support  activities,  such  as  maintenance.  The  maintenance  of  a  vehicle  can  be 
expressed  in  terms  of  the  metrics  listed  in  Table  10. 


Table  10  Maintenance  Metrics 

Metric 

Units 

Description 

Mean  Time  Between 

The  fleet-wide  average  length  of  time  between  successive  maintenance 

Maintenance  Actions 

hours 

actions,  which  informs  the  length  of  missions  and  deployments  (Ref.  26). 

(MTBMA) 

Mean  Time  to  Repair 

(MTTR) 

hours 

The  fleet-wide  average  length  of  time  required  to  perform  a  maintenance 

action  (Ref.  26). 

Maintenance  Man- 

The  fleet-wide  average  number  of  hours  spent  on  maintenance  actions  and 

Hours  per  Flight  Hour 

hours/hour 

inspections  required  for  each  flight  hour  flown  in  the  current  environment  (Ref. 

(MMH/FH) 

27). 

Cost  Flight  Hour 

(cost/FH) 

$/hour 

The  average  recurring  cost  for  each  flight  hour.  This  figure  can  be  broken  down 

into  consumables,  material,  labor,  and  facilities  (Ref.  28). 

Excess  Availability 

0/ 

The  proportion  of  time  that  the  vehicle  is  fully  operational  but  not  in  use  (Ref. 

/o 

28). 

The  objectives  for  any  technology  are  to  maximize  MTBMA,  while  minimizing  MTTR,  MMH/FH, 
and  cost/FH.  Excess  availability  should  meet  some  desired  threshold;  as  additional  capability  does 
not  add  value  to  the  system. 

Contemporary,  top-down  assessment  of  these  attributes  rely  on  close  interactions  with 
technologists  and  subject  matter  experts,  and  are  usually  expressed  in  a  qualitative  format.  A 
more  suitable  approach  is  to  create  a  bottom-up  model  for  vehicle  estimates  based  on  low-level 
technology  effects  that  can  be  determined  from  prototyping  or  literature  review. 
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2A.2.2  Conceptual  Model 

The  model  represents  a  fleet  of  rotorcraft  staging  from  a  forward  operating  base.  Vehicles  are 
passed  through  an  operational  cycle,  flying  missions  and  undergoing  inspection  and  maintenance. 
The  conceptual  flowchart  is  shown  in  Figure  26. 


Figure  26  Conceptual  Model  Flowchart 


Vehicle  operations  and  support  activities  occur  through  discrete  maintenance,  inspection,  and 
replacement  actions  that  are  either  time-based  (i.e.  the  replacement  of  a  life-limited  part  or 
scheduled  routine  inspection),  or  trigger-based  (i.e.  post-flight  inspection  or  the  repair  of  a  part 
damaged  on  mission)  (Ref.  29).  Therefore,  a  dynamic,  discrete  modeling  methodology  is  required. 
Trigger-based  actions  also  introduce  stochastic  effects  on  inspection  and  maintenance,  which 
must  be  reproduced  in  the  model. 

2.4. 2. 3  Literature  Review 

A  survey  of  dynamic,  discrete  modeling  methods  yielded  three  alternatives:  Markov  chains,  Petri 
nets,  and  Discrete  Event  Simulation  (DES).  Discrete  Event  Simulation  (DES)  was  ultimately  chosen 
as  it  condenses  the  simulation  time  by  only  performing  calculations  when  a  time-based  or  event- 
based  trigger  is  met,  as  opposed  to  a  continuous  system  which  must  calculate  every  time  step. 
Events  are  added  to  a  list,  which  is  stepped  through  chronologically  in  order  to  activate  the 
relevant  portions  of  the  model.  The  tools  examined  for  DES  construction  allowed  the  modeling  of 
multiple  types  of  tokens  moving  through  the  model,  such  as  resources  and  personnel.  Thus, 
Discrete  Event  Simulation  was  found  to  satisfy  the  modeling  requirements  identified  within  the 
conceptual  model. 

Discrete  Event  Simulation  is  a  well-known  simulation  tool  that  has  long  been  used  for  military 
systems  (Ref.  30).  Prior  work  utilizing  DES  for  vehicle  operations  have  focused  on  evaluating  or 
optimizing  operational  methodologies,  such  as  logistics  (Ref.  31)  or  maintenance  schedules  (Ref. 
32),  while  treating  the  vehicle  as  the  fundamental  object  within  the  simulation.  This  level  of 
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simulation  still  lacks  the  ability  to  model  part-level  technology  effects,  because  these  models  work 
on  failure  and  repair  rates  that  are  abstracted  to  the  vehicle-level. 

As  part  of  their  modeling  and  simulation  effort,  Arruda  et.  al.  created  a  DES  to  model  the  full- 
spectrum  operations  of  a  rotorcraft  fleet  (Ref.  33).  The  model  was  developed  in  SimPy,  an  open- 
source  module  for  Python.  Arruda  et.  al.  primarily  used  the  DES  to  evaluate  the  effects  of  active 
rotor  technologies  on  fleet  availability  and  maintenance.  Technology  factors  for  vibration  and 
noise  were  varied  on  a  baseline  vehicle  to  represent  these  effects.  Operational  metrics  were 
found  by  simulating  the  damage  and  repair  of  individual  vehicle  components.  The  work  reported 
in  this  paper  re-uses  this  basic  DES  environment  and  builds  upon  it.  This  work  retains  the  ability 
to  model  technology  impacts  on  a  fleet  of  vehicles,  while  expanding  the  scope  of  the  simulation 
from  a  single  configuration  with  fixed  vehicle  components  in  order  to  enable  freeform  vehicle 
configuration. 


2. 4.2. 4  Technical  Approach 

The  simulation  is  designed  for  a  deployment  of  a  rotorcraft  at  a  forward  operating  base.  The 
vehicle  is  passed  through  an  operational  cycle,  flying  missions  and  undergoing  inspection  and 
maintenance.  This  model  is  shown  in  Figure  27. 


{  Store  ^  [^Resource] 


Process 


Figure  27.  Discrete  event  simulation  model  flowchart  showing  stores,  resources,  and  processes 
The  vehicles  pass  between  stores,  represented  as  hexagons  in  Figure  4,  as  directed  by  the  processes, 
represented  as  rectangles.  The  Mission  Manager  manages  transitions  out  of  the  Vehicle  Depot, 
Inspection,  and  On  Mission  stores.  The  Maintenance  Manager  manages  transitions  out  of  the  Awaiting 
Maintenance  and  Maintenance  stores.  At  the  start  of  the  simulation,  input  variables  are  instantiated  for 
the  base,  as  listed  in  Table  11.  The  simulation  is  executed  for  a  set  amount  of  time,  and  runs  until  that 
time  has  expired  or  no  new  events  are  created. 
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Table  11.  Base  Input  Variables 


Input 

Units 

Description 

Number  of  Pilots 

The  number  of  pilots  available  for  a  mission 

Pilot  Burden/FH 

$/hours 

The  direct  cost  of  supporting  a  pilot  for  one  flight  hour 

Fuel  Cost 

$/gallon 

The  cost  of  aviation  fuel 

Number  of  maintenance 

The  number  of  maintenance  personnel  available 

personnel 

Maintenance  Start 

Time  of 

Time  when  the  maintenance  shop  opens  each  day  and  becomes 

day 

available  for  activities 

Maintenance  End 

Time  of 

Time  when  the  maintenance  shop  closes  each  day  and  can  no  longer 

day 

perform  activities. 

Number  of  parts 

The  quantity  of  each  type  of  part  stored  at  the  base 

2.4. 2. 5  Vehicles  &  Parts 

Each  vehicle  is  constructed  as  a  container  that  is  filled  with  parts.  Parts  can  be  created  for  any 
system  on  the  vehicle,  such  as  an  engine,  a  rotor  blade,  or  landing  gear.  At  the  start  of  the 
simulation,  part  objects  are  created  and  populated  with  values  drawn  from  the  input  files.  The 
part  objects  are  then  allocated  to  vehicle  objects.  All  parts  reside  on  the  same  level  of  hierarchy 
within  the  system. 

Many  different  configurations  can  be  modeled  by  adding  the  relevant  parts  because  the  vehicle 
is  built  from  the  component-level.  A  single  main  rotor  can  be  modeled  by  including  one  main  rotor 
part,  or  a  tilt-rotor  can  be  modeled  by  adding  two  props  and  the  associated  tilt  mechanisms. 

The  use  of  parts  can  also  allow  different  levels  of  simulation,  depending  on  the  detail  of  the  parts. 
At  the  highest  levels,  parts  can  act  as  entities  like  'airframe,'  'rotor,'  and  'engine,'  but  these  can 
be  decomposed  into  parts  such  as  individual  rotor  blades,  structural  linkages,  and  shafts,  to  suit 
the  model  requirements.  Given  the  availability  of  data,  the  modeler  can  choose  to  create 
components  at  any  desired  level  of  detail. 

Each  part  tracks  the  flight  hours,  damage,  and  total  cost  incurred  throughout  its  lifetime.  These 
metrics  are  calculated  and  updated  according  to  the  part's  input  value  types,  listed  in  Table  12. 
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Table  12.  Part  Input  Variables 


Input 

Units 

Description 

MTTR 

hours 

The  mean  time  to  repair  the  part,  including  uninstallation  from  the  vehicle, 

repair  to  the  component  or  replacement,  and  reinstallation. 

Inspection  Time 

hours 

The  time  required  to  perform  a  routine  inspection  on  the  component  without 

removal. 

Inspection 

Threshold 

% 

The  cumulative  damage  threshold  that  will  trigger  component  repair  or 

replacement. 

Lifespan 

hours 

The  flight  hours  allowed  for  a  life-limited  component. 

Unit  Cost 

k$ 

The  purchase  cost  for  a  new  component. 

Repair  Cost 

k$ 

The  mean  cost  in  material  to  repair  the  component,  excluding  unit  cost  and 

labor. 

Failure  Mode 

'flight'  or  'fatigue' 

Enumeration  to  determine  how  the  model  will  track  part  usage  and  trigger 

maintenance. 

Failure  Effect 

'abort' 

'loss'  or 

'continue' 

Enumeration  that  determines  the  effect  of  a  part  failure  during  flight. 

Fatigue 

Properties 

Stress  as  a  function  of  cumulative  number  of  cycles,  based  on  an  aggregate  of 

the  part,  or  based  on  a  limiting  material. 

The  simulation  also  tracks  the  number  of  times  maintenance  is  performed  on  each  part  type.  At 
the  system  level,  the  simulation  tracks  each  vehicle's  flight  hours  and  time  since  previous 
maintenance  activity. 

2.4. 2. 6  Mission  Manager 

The  Mission  Manager  determines  when  missions  are  generated,  and  randomly  draws  the  mission 
from  a  weighted  list  of  mission  types  created  by  the  user.  Each  mission  is  created  as  a  set  of 
phases;  an  example  is  shown  in  Table  13. 


Table  13.  Phase  Definition  for  a  Combat  Mission 


Phase 

Duration 

Altitude 

Temp 

Payload 

Flover 

2.5  min 

2500  ft 

59  “F 

1000  lb 

Cruise 

100  min 

3000  ft 

59  “F 

1000  lb 

Loiter 

15  min 

3000  ft 

59  “F 

1000  lb 

Cruise 

100  min 

3000  ft 

59  “F 

1000  lb 

Flover 

2.5  min 

2500  ft 

59  “F 

1000  lb 
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Phases  can  be  arranged  to  represent  different  types  of  missions,  such  as  attack,  transport, 
scouting,  or  medevac.  The  mission  is  then  assigned  to  an  available  vehicle  in  the  Vehicle  Depot  at 
which  time  the  vehicle  undergoes  preflight  inspection  and  is  allocated  pilots  and  fuel. 

The  Mission  Manager  may  also  trigger  any  outstanding  maintenance  items  the  vehicle  has 
accumulated.  For  instance,  if  the  vehicle  is  inspected  before  the  mission,  and  a  part  is  found  to  be 
damaged  or  have  exceeded  its  lifespan,  the  vehicle  will  be  moved  to  maintenance,  and  another 
vehicle  substituted  on  the  mission. 

The  mission  includes  performance  calculation  and  part  damage  calculation.  Equations  from 
Leishman  (Ref.  34)  are  used  to  calculate  fuel  burn  and  vehicle  g-loading  for  the  selected  mission. 
The  vehicle  speed  is  then  used  as  the  input  to  a  vibration  map  in  order  to  generate  the  vibration 
amplitude  for  use  in  part  damage  calculations.  The  vibration  map  was  created  as  a  regression  on 
historical  UH-60  information 

(Ref.  35).  If  a  part's  fatigue  characteristics  are  supplied,  part  damage  can  be  calculated  via  Miner's 
rule  for  cumulative  damage  (Ref.  36): 

=  £f=i  niSi  (!) 

NS 

Vibration  and  loading  are  calculated  for  each  mission  phase  flown,  and  the  results  are  used  to 
increase  the  part's  damage.  Otherwise,  the  part  lifespan  is  calculated  directly  from  the  input  value 
for  lifespan. 

If  a  part  exceeds  its  damage  threshold,  the  part  ends  the  mission  according  to  its  failure  effect. 
Missions  can  be  successfully  completed  with  or  without  damage,  aborted  with  damage,  or  the 
vehicle  can  be  lost.  If  the  mission  is  successfully  completed  or  aborted,  the  Pilots  are  returned  to 
the  resource  store.  Consumables  such  as  fuel  and  parts  are  not  recycled. 

2.4. 2. 7  Maintenance  Manager 

The  Maintenance  Manager  determines  whether  the  vehicle  requires  maintenance.  If  so,  the 
vehicle  is  queued  for  the  relevant  maintenance  actions,  and  is  assigned  parts  and  mechanics  as 
they  become  available.  Once  the  action  is  complete,  the  vehicle  returns  to  the  Vehicle  Depot. 
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Vehicle-level  maintenance  actions  are  categorized  into  two  groups:  scheduled  and  unscheduled 
maintenance.  Scheduled  maintenance  is  triggered  after  a  number  of  flight  hours,  or  in  set  time 
intervals.  Unscheduled  maintenance  involves  repair  or  replacement  of  a  part  due  to  damage 
sustained  during  a  mission.  Maintenance  actions  for  each  part  can  currently  be  triggered  in  one 
of  two  ways:  damage,  or  exceeding  lifespan.  Replacement  or  repair  can  be  triggered  simply  by  the 
part's  total  flight  hours  exceeding  its  lifespan.  Alternatively,  given  information  about  the  part's 
fatigue  characteristics,  replacement  can  be  triggered  by  damage.  This  method  assumes  that  the 
damage  will  be  revealed  by  an  inspection  when  the  inspection  threshold  has  been  reached,  and 
the  part  will  fail  once  the  damage  reaches  1.0.  The  replacement  by  damage  is  designed  to  allow 
evaluation  of  active  health  monitoring  (AHM)  technologies,  such  as  HUMS.  This  approach  is  best 
suited  to  parts  that  have  a  characteristic  material  that  is  most  susceptible  to  fatigue,  such  as  a 
driveshaft  or  rotor  blade  linkage,  rather  than  complex  components  such  as  avionics. 

2.4. 2.8  Cost  Modeling 

Direct  operating  costs  for  each  mission  are  calculated  using  labor  burden  for  pilots  and  the  cost 
of  consumables,  as  shown  in  Equation  2: 

Bmiss  =  RH  *  Bp  +  Ccons  (2) 

At  the  completion  of  the  simulation,  each  mission  cost  is  summed  and  the  average  cost  per  flight 
hour  found.  Maintenance  costs  for  each  activity  are  calculated  by  Equation  3: 

Cmaint  =  MTTR*Bm  +  Cmat  (3) 

In  addition,  the  inspection  costs  are  calculated  by  using  Equation  4: 

Rinsp  —  Rinsp  *  Bm  (4) 

Average  maintenance  costs  are  found  by  summing  the  maintenance  and  inspection  costs,  and 
averaging  them  across  the  total  flight  hours  for  the  fleet.  Upon  completion  of  the  simulation, 
fleet-wide  metrics  for  cost  and  time  are  calculated  by  summing  the  data  stored  by  each  vehicle 
and  part. 

2.4. 2.9  Technology  Representation 

This  model  has  the  capability  of  representing  technologies  that  impact  vehicle  parts, 
performance,  and  base  maintenance  practices  through  the  use  of  impact  factors  or  "k-factors". 
K-factors  represent  the  impact  a  technology  has  on  different  characteristics  in  the  model  as  a 
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scaling  factor,  i.e.  a  percent  change  from  the  baseline.  For  example,  if  a  new  technology  is 
expected  to  reduce  vibrations  in  the  vehicle  by  five  percent,  it  may  be  represented  by  applying  a 
k-factor  on  the  result  of  the  vibration  calculation.  A  k-factor  of  -0.05  and  would  result  in  the 
following  relationship: 

Xnew  =  0.95Xold  (5) 

By  using  k-factors  it  is  therefore  unnecessary  to  customize  the  code  to  incorporate  the  exact 
mechanism  that  resulted  in  the  lower  vibrations. 

In  the  model,  performance  factors  include  those  impacting  power,  empty  weight,  figure  of  merit, 
lift-to-drag  ratio,  and  vibration  magnitude,  matching  the  technology  factors  available  in 
quantitative  performance  codes  (Ref.  37).  Additionally,  each  part  input  can  be  modified  to 
account  for  the  expected  impact  of  a  technology  on  its  maintenance  parameters.  New  parts  may 
also  be  added  to  change  the  configuration  to  reflect  the  use  of  a  technology.  K-factors  are  set 
within  the  model  input  files  on  each  part  type  and  on  the  base  inputs.  Representing  multiple 
technology  effects  is  achieved  by  externally  calculating  an  overall  k-factor  for  each  parameter, 
and  substituting  that  into  the  model. 

2.4.3  Model  Verification 

In  order  to  verify  the  model,  a  proof-of-concept  model  instance  was  created  for  a  fleet  of  six  UH- 
60M  vehicles.  This  vehicle  was  chosen  as  the  baseline  due  to  readily  available  data  and  its 
relevance  to  the  research  objectives.  Vehicle  performance  figures  used  in  the  mission  calculation 
were  taken  from  an  NDARC  UH-60M  model.  As  stated  previously,  within  the  model  a  vehicle  can 
be  represented  with  as  many  or  as  few  parts  as  desired.  The  model  vehicle  was  built  from  the 
parts  listed  in  Table  14  and  is  flown  through  the  basic  mission  described  in  Table  13.  These  parts 
and  mission  were  selected  to  allow  for  an  adequate  representation  of  the  UH-60M  baseline. 


Table  14.  Vehicle  Components 


Forward  Airframe 

Mission  Equipment 

Auxiliary  Power  Unit 

Center  Airframe 

External  Supports 

Hydraulics 

Aft  Airframe 

Avionics 

Fuel  System 

Tail  Pylon 

Engine 

Flight  Controls 

Electronics 

Main  Rotor 

Transmission 

Landing  Gear 

Control  Rod 

Tail  Rotor 
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The  simulation  was  performed  by  switching  maintenance  calculations  for  the  vehicle  parts 
between  life-limited  and  AHM.  AHM  is  simulated  in  the  model  by  initiating  maintenance  actions 
based  on  the  part  reaching  a  threshold  on  its  cumulative  damage.  Additionally,  a  sweep  was  made 
on  the  mission  rate  per  day. 


2.4.3.1  Results 

Notional  results  from  the  simulation  relating  to  excess  availability  are  shown  in  Figure  28.  The 
dominant  trend  of  the  figure  illustrates  the  inverse  relationship  between  mission  rate  and  excess 
availability.  As  more  missions  are  generated,  vehicles  spend  more  time  on  missions  and  in 
maintenance,  and  less  time  waiting.  Switching  between  life-limited  maintenance  and  AHM  has 
the  effect  of  increasing  the  excess  availability,  due  to  the  increased  MTBMA. 

0.055 
0.05 
0.045 
0.04 
#  0.035 
;§  0.03 
3  0.025 
0.02 
0.015 
0.01 
0.005 

123456789 
Sortie  Rate 

Figure  28.  Preliminary  Results  for  Fleet  Excess  Availability  vs.  Mission  Rate 


The  inverse  trend  is  expected  intuitively  and  confirmed  by  the  same  relationship  found 
analytically  by  Scott  in  Reference  22.  While  the  work  of  Scott  focuses  on  civilian  rotorcraft, 
scheduled  flight  hours  per  year  and  mission  rate  are  both  measures  of  demand,  and  therefore  the 
same  trend  is  expected.  The  asymptote  observed  in  Figure  28  is  the  result  of  the  mission  manager 
aborting  missions  due  to  a  lack  of  available  vehicles.  Vehicles  that  would  have  become  ready 
partway  through  a  mission  are  instead  available. 


Further  analysis  of  the  data  showed  another  trend.  As  the  maintenance  calculation  is  switched  to 
AHM  from  life-limited,  the  required  MMH/FH  for  the  fleet  decreases,  as  demonstrated  in  Figure 
29. 
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Figure  29.  Preliminary  Results  for  Fleet  MMH/FH  vs.  Mission  Rate 

When  the  results  shown  in  Figure  28  and  Figure  29  are  evaluated  together,  it  can  be  seen  that 
when  utilizing  AHM  the  excess  availability  is  increased  while  simultaneously  decreasing  MMFI/FH. 

2A.3.2  Concluding  Remarks 

The  DES  model,  as  executed  in  the  proof  of  concept,  demonstrated  results  that  could  not  be 
quantified  from  performance-based  tools  such  as  NDARC.  A  test  case  using  the  DES  model 
demonstrated  the  ability  to  assess  a  technology  that  impacts  non-performance  attributes,  giving 
a  more  complete  picture  of  the  capability  space  for  advanced  rotorcraft. 

2.4.4  Critical  Questions  and  Limitations 

While  preparing  the  model  for  presentation  at  AFIS  Forum  72,  a  number  of  limitations  were  identified 
that  needed  to  be  addressed  going  forward.  The  first  limitation  identified  had  to  do  with  the  Mission 
Manager  and  how  the  vehicle  is  passed  to  maintenance.  Prior  to  sending  a  vehicle  on  a  sortie,  the 
Mission  Manager  performs  a  preflight  inspection.  During  preflight  inspection  if  there  are  any  damages 
then  the  vehicle  is  passed  to  the  Maintenance  Manager.  However,  the  Mission  Manager  also  assesses 
whether  the  mission  will  cause  a  failure  and  in  anticipation  of  this  will  send  the  vehicle  to  preventative 
maintenance  prior  to  failure.  The  critical  result  of  this  is  that  the  vehicle  never  suffers  a  mission  critical 
failure. 

The  next  limitation  identified  had  to  do  with  the  randomness  of  the  model.  In  order  to  evaluate  the 
RAM  metrics  identified  in  the  model  description,  the  model  needs  to  iterate  on  a  stochastic  process 
and  average  the  results.  This  is  currently  not  the  case  as  the  model  takes  in  fixed  point  values  for  all 
inputs  and  as  a  result  the  output  is  inherently  deterministic. 
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The  last  limitation  identified  prior  to  presenting  was  that  the  model  doesn't  capture  several  of  the 
Future  Vertical  Lift's  desired  metrics.  The  model  does  not  currently  report  on  the  Maintenance  Free 
Operating  Period  (MFOP)  and  False  Removal  Rate  (FRR)  for  the  vehicle. 

In  addition  to  the  limitations  identified  prior  to  presenting  the  model  at  AHS  Forum  72,  the  feedback 
from  the  audience  identified  several  critical  questions  that  should  be  addressed.  Firstly,  the  method 
for  aging  the  parts  is  currently  modeled  after  a  technique  presented  in  the  Principles  of  Helicopter 
Aerodynamics  text  by  Leishman.  The  referenced  vibration  mapping  technique  used  to  age  the  vehicle 
in  the  current  version  of  the  model  is  dated,  circa  1970s,  and  is  likely  inaccurate  for  this  era  of  vehicles. 
A  more  accurate  form  of  aging  the  vehicle  is  necessary  if  the  outputs  are  ever  to  be  validated. 
Secondly,  not  all  components  are  critical  for  each  mission  phase  and  in  addition  to  that  not  all 
components  are  safety  critical  some  are  just  mission  critical.  In  other  words,  while  a  component  may 
be  critical  for  both  the  safety  of  the  vehicle  and  the  success  of  the  mission,  another  less  critical 
component  may  be  only  mission  critical,  having  no  effect  on  the  safety  of  the  vehicle.  Expanding  this 
question,  consideration  should  be  given  to  how  the  critical  components  vary  with  respect  to  the 
mission  phase. 

The  following  table,  Table  15,  breaks  down  each  aforementioned  question  or  limitation  and  how  it  is 
being  addressed. 


Table  15:  Operations  Model  Limitations 


Question/Limitation 

Addressed  in  FY16 

Not  Addressed 

1.  With  preflight  inspection  and  failure 

Corrected  with  IPLE 

anticipation  the  vehicle  never  suffers  a 

operations  model 

mission  critical  failure 

2.  The  model  is  inherently 

deterministic,  there  is  no 

stochastic  process  representing 

what  is  realistically  happening 

IPLE  model  is  built  on 

stochastic  inputs,  new 

Maintenance  Manager 

takes  stochastic  input  for 

MTTR,  Cost,  and  Shop 

Status 
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3.  Maintenance  Free  Operating 

Period  (MFOP)  and  False  Removal 

Rate  (FRR)  are  not  considered 

MFOP  capability  added 

during  IPLE  merge 

FRR  could  be  considered,  no 

framework  exists  currently 

4.  The  Leishman  vibration  mapping  is 

outdated,  circa  1970s,  and  is  likely 

inaccurate  for  this  era  of  vehicles 

By  using  component  failure 

rate  data,  there  is  no  need 

for  vibration  mapping 

5.  There  is  no  consideration  given  to 

mission  critical  and  safety  critical 

components  being  represented  in 

the  operational  cycle  nor  how 

these  vary  between  mission 

phases 

The  IPLE  model  is  built  on 

fault  trees  input  for  each 

unique  mission  phase.  One 

for  mission  critical  and  one 

for  safety  critical. 

6.  Plan  for  validating  the  model 

There  is  currently  no  plan  to 

address  this  as  doing  so 

requires  field  data 

2.4.5  Improved  Model  with  Phased-Mission  Simulation 

To  study  the  Reliability,  Availability,  and  Maintainability,  an  integrated  simulation  environment  was 
developed,  which  is  illustrated  in  Figure  30.  Detailed  description  of  the  various  steps  in  the  simulation  are 
described  in  the  following  sub-sections.  This  discrete-event  simulation  program  is  designed  to  be  modular 
and  represent  the  fact  that  each  operational  mission-phase  has  a  different  set  of  mission/safety  critical 
systems  as  well  as  systems  in  use.  The  simulation  is  performed  for  a  single  vehicle;  the  simulation  assumes 
a  certain  number  of  flight  hours  the  vehicle  will  be  in  operation  for  and  once  this  number  is  reached,  one 
monte-carlo  run  is  completed.  A  similar  full  operation  cycle  is  run  multiple  times  to  obtain  the  long-term 
steady-state  results  for  the  various  simulation  metrics,  such  as  overall  mean  up-time  and  down-time,  etc. 
The  simulation  requires  a  typical  mission  to  be  inputted  with  its  different  phases  and  time  spent  in  each 
phase,  such  as  Hover,  Cruise,  etc.  Each  phase  has  a  set  of  mission-critical  systems,  modeled  as  event  trees 
(this  is  described  in  section  2. 4. 5.1).  These  mission  critical  systems  are  encoded  through  a  fault-tree:  each 
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mission  phase  has  its  own  unique  event-tree.  Similarly,  each  mission  phase  will  have  its  own  safety  critical 
system  list  (fault-tree).  The  simulation  input  file  has  a  table  of  information  with  mission  phases,  the 
mission  and  safety  critical  fault-trees,  system/component  reliabilities,  etc.  Each  phase  also  has  a  list  of 
systems/components  that  'age'  (this  is  described  in  section  2. 4. 5. 2). 

The  simulation  uses  random  variates  to  model  the  different  component  reliabilities  for  simulating  system 
failures  and  keeps  track  of  component-age,  and  maintenance  actions  can  be  scheduled  to  reset  the  age 
of  failed  and  repaired  systems,  accordingly.  This  feature  adds  stochastic  variability  to  the  simulation, 
addressing  the  deterministic  issue.  When  a  mission-critical  event  occurs  in  the  simulation,  the  rest  of  the 
mission  is  aborted  and  the  vehicle  is  sent  into  maintenance.  When  the  vehicle  goes  into  maintenance,  the 
maintenance  manager  computes  the  down-time  based  on  the  Mean  Time  To  Repair  (MTTR), 
Administrative  Delay  Time  (ADT),  and  Logistics  Delay  Time  (LDT),  and  maintenance  costs— these  are  all 
currently  modeled  with  stochastic  variability  as  well. 

The  simulation  can  be  run  in  different  configurations  to  either  estimate  Operational  Availability*  (A0), 
System  Safety,  or  Maintenance  Free  Operating  Period  (MFOP),  etc.  The  focus  of  the  current  research  has 
been  on  predicting  Operational  Availability  and  O&S  cost. 


Operational  Availability  is  defined  as  the  ratio  of  'Uptime'  to  the  sum  of  'Uptime  and  Downtime'. 


Stochastic  Petri-Net 
Simulation 


Availability/ 

MTMAF 

Distribution 


Figure  30.  Integrated  Discrete  Event  Simulation  Environment 


Safety 

Distribution 
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2. 4. 5.1  Mission  Critical  Event  Tree 

A  mission  critical  event  tree  is  similar  to  a  fault  tree  structure  and  based  on  what  mission-phase  the 
vehicle  is  in,  a  series  of  mission-critical  events  can  be  modeled.  An  event  tree  automatically  generated 
in  Simulink  is  shown  in  Figure  33.  This  event  tree  is  created  in  excel,  as  shown  in  Figure  32.  This  process 
requires  some  a-priori  knowledge  of  the  system  architecture  and  how  the  system  behaves  in  different 
missions.  In  conceptual  design  stages  this  process  requires  the  usage  of  some  historical  information 
of  system  architecture  and  failure  information,  as  the  design  process  progresses  and  more 
information  is  made  available,  this  should  be  updated. 

2.4. 5.2  System  Ageing 

A  unique  feature  of  this  simulation  model  is  the  ability  to  treat  certain  systems/components  as 
dormant  during  certain  mission  phases.  This  means  that  these  systems  will  have  their  own  ageing 
clock  that  will  allow  ageing  only  in  mission  phases  that  utilize  this  system.  For  example:  the  landing 
gear  will  not  age  during  cruise  mission  phase.  This  aspect  of  the  program  also  allows  for  resetting  the 
age  of  repairs/replaced  components  without  affecting  other  systems/components.  This  is  another 
reason  for  breaking  down  the  systems  to  the  component  level,  otherwise  the  simulation  will 
erroneously  reset  the  age  of  the  top-level  system  instead  of  the  repaired/replaced  component.  These 
features  are  extremely  important  to  accurately  simulate  the  actual  operation  and  functioning  of  the 
vehicle.  For  these  reasons,  the  phased-mission  and  ageing  component  form  of  simulation  is  more 
accurate  than  a  general  overall  system  petri-net. 


2. 4. 5. 3  Input  Process 

Efforts  have  been  taken  to  improve  the  usability  of  the  previously-developed  model  by  increasing 
input  scalability,  implementing  input  error-checking  capabilities,  and  improving  the  Excel  input  user 
interface.  A  flow-chart  of  the  process  for  input  and  error-checking  is  shown  in  Figure  31. 
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Segment 
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Figure  31.  Input  Data  Flow 
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The  Excel  input  user  interface  for  one  mission  phase  with  a  mission  critical  fault  tree  is  shown  below 
in  Figure  32.  This  template  is  capable  of  accepting  fault  trees  for  safety  critical  and  mission  critical 
failures.  Each  mission  phase  is  represented  by  a  different  sheet  in  Excel,  increasing  the  flexibility  of 
the  modeling  approach  as  an  appropriate  fault  trees  may  be  implemented  for  each  mission  phase.  In 
the  example  shown,  only  the  mission  critical  fault  tree  is  included  for  each  phase.  The  'Check  Fault 
Tree'  button  allows  the  user  to  check  the  fault  tree  for  errors  before  running  the  operations  model. 


Top  Cell  Only,  Too  many  systems  specified  in  that  column 

Whole  Column  =  Not  Enough  systems  specified  in  that 
column 

Number  of  additional  cells  indicates  number  of  missing 
systems 

Known  Failure  Distribution 

-1 

'OR'  Statement 

0 

'AND'  Statement 

No  leading  value 

Check  Fault  Tree 
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4 
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-1 
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0,2,15,9,3 

-1 

0,22,12 

-1 

-1 

0,24,25,26,27 

-1 

-1 

3,4 

-1 

-1 

0,34,18,33 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 
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-1 

-1 

-1 

-1 

-1 

-1 

-1 

-1 

Safety  Critical  *0,0 


Figure  32.  Excel  Input  Template  for  Fault  Trees 

Fault  trees  are  represented  using  a  top-down  approach  looking  left  to  right  in  Figure  32.  Cells 
corresponds  to  either  an  OR  gate  (a  cell  with  a  leading  zero)  and  corresponding  inputs  for  the  gate, 
an  AND  gate  (a  cell  with  no  leading  zero)  and  corresponding  inputs  for  the  gate,  or  a  -1  that  tells  the 
fault-tree  reading  script  to  use  a  failure  distribution  for  the  component  as  defined  in  the  'Systems' 
tab.  Any  'G'  represents  an  intermediate  gate  and  is  used  for  additional  system  decomposition  and 
may  correspond  to  either  an  AND  or  an  OR  gate.  As  systems  or  intermediate  gates  are  added, 
additional  definition  is  provided  in  the  next  column.  A  column  is  organized  by  reading  left  to  right  in 
the  first  through  the  last  cells  of  the  previous  row.  As  an  example,  look  at  the  third  column  of  Figure 
32.  In  the  first  row,  the  entry  is  '0,G8,G9,G10,23,25'.  This  is  an  OR  gate  with  3  intermediate  gates:  G8, 
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G9,  and  G10;  and  2  specific  systems  defined:  23  and  35.  In  the  next  column  the  three  gates  are 
defined,  which  all  happen  to  be  additional  OR  gates,  and  the  two  systems  have  -1  values  in  the  next 
column  that  tell  the  program  to  look  for  a  failure  distribution  for  those  components.  Using  this 
approach,  the  modeling  philosophy  is  flexible  and  scalable  and  may  include  any  size  fault  tree  or 
number  of  components. 

Since  the  fault  trees  are  a  critical  component  of  the  operational  analysis,  it  is  important  that  the  fault 
trees  used  in  the  operations  analysis  match  user  expectations.  The  simplest  way  to  verify  that  the 
fault  trees  entered  into  the  Excel  sheet  are  consistent  with  user  expectations  is  by  visual  inspection 
of  the  fault  tree.  The  fault  trees,  as  represented  in  Excel,  are  read  into  the  operations  model  as  a  cell 
array  using  a  MATLAB  script.  This  cell  array  is  used  to  automatically  create  a  visual  representation  of 
the  fault  trees  in  Simulink.  Because  no  features  exist  to  rearrange  blocks  for  aesthetics  and  a  general 
formulation  is  difficult  to  generate,  connections  between  blocks  may  cross  over  other  blocks  at  times. 
An  automatically  generated  fault  tree  corresponding  to  the  Excel  input  shown  in  Figure  32  is  shown 
in  Figure  33  below.  Note  that  this  diagram  is  intended  to  be  a  second-level  of  error  checking  to  ensure 
that  the  fault  tree  is  consistent  with  the  intent  of  the  user. 


Figure  33.  Simulink  Fault  Tree  Diagram  Automatically  Generated  for  Visual  Error  Checking 

2. 4. 5. 4  Preliminary  Results 

The  following  figures  are  simulation  results  obtained  by  simulating  1500  flight  hours  for  a  notional  utility 
mission  with  some  nominal  failure/repair  rates,  with  only  one  non-unique  mission-critical  event  tree 
(shown  in  Figure  33).  The  monte-carlo  runs  are  set  to  terminate  when  the  coefficient  of  variation  between 
runs  reaches  a  certain  threshold.  These  results  are  presented  to  demonstrate  capability  of  the  operations 
model  and  are  preliminary  results  that  are  not  fully-indicative  of  an  actual  vehicle. 
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The  raw  data  for  time  between  mission  affecting  failures,  for  all  simulation  runs  is  shown  in  Figure  34.  This 
figure  shows  that  for  no  instance  of  the  simulation,  was  the  vehicle  able  to  perform  over  12  hours  of 
operation  without  requiring  to  abort  mission  due  to  some  component  failure.  The  inverse  cumulative 
distributive  function  for  Operational  Availability  (A0)  is  shown  in  Figure  35,  and  given  the  aforementioned 
limitations  of  this  simulation,  the  results  are  narrowly  distributed  between  39.5%  and  43.5%.  Similarly, 
the  Mean  Time  Between  Mission  Affecting  Failures  (MTBMAF)  is  shown  in  Figure  36;  this  is  raw  data 
aggregated  for  each  monte-carlo  run.  According  to  this  plot,  the  MTBMAF  for  this  vehicle  will  neither  be 
greater  than  3.1  hours  nor  less  than  2.75  hours.  The  simulation  is  also  built  with  the  capability  to  identify 
mission  abort  influencing  components,  and  this  is  plotted  as  histogram  shown  in  Figure  37.  Since  the 
simulation  uses  aggregated  systems,  some  systems  such  as  'Airframe'  and  'Main  Rotor  Installation'  show 
an  unusually  high  failure  frequency— these  results  can  be  made  more  accurate  and  realistic  by  including 
more  descriptive  mission 
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Figure  34.  Raw  Data  CDF  of  Time  Between  Mission  Affecting  Failures 
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Availability  Inverse  CDF:  95%=0.40635  75%=0.41287 


MTBMAF  Inverse  CDF:  95%=2.8454  75%=2.9034 


Figure  36.  Inverse  CDF  of  Mean  Time  Between  Mission  Affecting  Failures 
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Average  Number  of  Component  Failures  of  Each  Type  per  MC  Cycle 


Figure  37.  Histogram  of  System/Component  Failure  Frequency 


2.4.6  Future  Work 

The  team  will  continue  to  make  enhancements  to  this  simulation  environment  so  it  is  able  to  better 
predict  the  RAM  metrics  of  interest.  Some  areas  that  are  currently  being/potentially  be  pursued  for  FY17 
are: 

1.  System  and  component  redundancy:  adding  this  feature  to  the  model  will  help  study  component 
redundancy  trade-off  between  Availability  and  Capability. 

2.  Populating  mission-phase  specific  event  trees  and  component  ageing  information.  The  model 
currently  does  not  have  this  information  and  more  work  needs  to  be  done  in  this  area  to  be  able 
to  predict  RAM  data  accurately. 

3.  Adding  safety  critical  information  through  Function  Hazard  Assessments  (FHA)  and  Failure  Mode 
Effects  and  Criticality  Analysis  (FMECA)  would  add  capability  to  study  system  safety. 

4.  Adding  fidelity  to  maintenance  manager:  the  current  model  needs  to  be  enhanced  to  better 
predict  Mean  Down  Time  (MDT),  which  includes  repair  time,  logistics  delay  time,  and 
administrative  delay  time. 

5.  Exploring  maintenance  paradigms.  Different  maintenance  paradigms  could  be  explored  using  this 
simulation  to  study  how  RAM/cost  is  affected  in  the  different  cases.  Dynamically  scheduling 
maintenance  actions  on  certain  parts  is  an  example.  The  current  model  has  a  single  level 
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maintenance  paradigm  in  that  all  maintenance  occurs  at  a  single  hub.  Expanding  this  to  a  two 
level  paradigm  where  more  exhaustive  maintenance  occurs  at  the  OEM  or  maintenance  depot 
would  add  a  more  thorough  representation  of  O&S  costs  and  downtime. 

6.  Improving  O&S  cost  prediction.  Incorporating  more  data  on  component  cost  and  how 
maintenance  man-hours  can  be  modeled  will  give  better  prediction  of  O&S  cost. 


2.4.7  Combat  Survivability  Model 

The  addition  of  a  combat  survivability  model  to  the  current  operations  simulation  is  targeted  at 
allowing  the  evaluation  of  the  combat  effectiveness  of  different  concepts.  For  utility-class  rotorcraft, 
the  focus  of  the  current  work,  combat  effectiveness  can  be  assessed  simply  as  surviving.  Aircraft 
combat  survivability  is  defined  by  R.  Ball  in  The  Fundamentals  of  Aircraft  Combat  Survivability  Analysis 
and  Design,  Second  Edition  as  the  capability  of  an  aircraft  to  avoid  or  withstand  a  man-made  hostile 
environment  (Ball,  2003).  This  can  be  measured  as  the  probability  the  aircraft  survives  an  encounter 
(combat)  with  that  environment: 


Ps  ~  1  —  Pk  ~  1  —  PhPk\h 

Survivability  =  1  -  Killability  =  1  -  Susceptibility  *  Vulnerability 

Where  susceptibility  is  the  inability  of  an  aircraft  to  avoid  being  hit  by  the  environment  and 
vulnerability  is  the  inability  of  an  aircraft  to  withstand  that  hit.  It  is  logical  to  then  conclude  that  any 
technology  or  concept  which  reduces  the  vulnerability  and  susceptibility  of  the  aircraft  to  the  combat 
environment  will  increase  the  aircraft  survivability  and  ultimately  reduce  O&S  costs. 

Research  into  the  typical  threat  environments  encountered  by  utility-class  rotorcraft  by  mission  is 
detailed  in  Table  16.  It  can  be  seen  that,  unlike  fixed  wing  aircraft,  slow  moving,  noisy  and  relatively 
soft  vehicles,  such  as  rotorcraft,  operating  in  close  proximity  to  the  ground  and  hostile  ground  forces 
are  exposed  to  a  wide  range  of  threats.  While  some  discrepancies  occur,  such  as  the  presence  of 
Surface-to-Air  Missiles  (SAMs),  for  the  desired  fidelity  of  the  model  it  is  sufficient  to  approximate  the 
threat  environment  to  that  of  the  Air  Support  and  Battlefield  Insertion/Extraction  missions. 
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Table  16:  Threat  Environment  by  Mission 


Battlefield 

Insertion/ 

Extraction 

Urban  Insertion/ 
Extraction 

Special  Operations 

Humanitarian  Aid 

Air  Support 

Search  and  Rescue 
(SAR) 

Small  Arms 

X 

X 

X 

X 

X 

X 

Machine  Guns 

X 

X 

X 

X 

X 

X 

Self  Propelled  Anti- 

Aircraft  Gun 

(SPAAG)/Semi- 

X 

X 

X 

X 

Mobile  AAA 

Mortars 

X 

X 

X 

X 

X 

X 

Rocket  Propelled 
Grenade (RPG) 

X 

X 

X 

X 

X 

X 

Artillery 

X 

X 

X 

X 

X 

Man  Portable  Air 

Defense  System 

X 

X 

X 

X 

X 

X 

(MANPAD) 

Surface-to-Air 

Y 

Y 

Missile  (SAM) 

A 

A 

Model  Assumptions 

A  number  of  assumptions  are  made  to  simplify  the  model  while  still  maintaining  the  desired  level  of 
fidelity.  Most  notably  the  detectability  of  the  vehicle  is  assumed  to  be  100%.  This  assumption  is  made 
due  to  utility-class  rotorcraft  having  very  little  in  the  way  of  EO/IR  signature  reduction  and  noise 
control  in  addition  to  flying  low  and  slow.  Another  assumption,  as  mentioned  previously,  is  given  the 
consistency  of  rotorcraft  threat  environments  it  is  sufficient  to  approximate  the  threat  environment 
to  be  constant,  corresponding  to  that  of  the  Air  Support  and  Battlefield  Insertion/Extraction  missions. 
Along  the  same  line,  given  that  the  vehicle  being  modeled  is  utility-class,  it  is  assumed  that  the  vehicle 
has  no  offensive  capabilities  to  be  modeled.  Lastly,  to  simplify  the  analysis,  a  constant  speed  and 
altitude  corresponding  with  the  cruise  mission  phase  will  be  used. 

UH-60M  Use  Case 

Once  the  combat  survivability  methodology  is  complete,  a  case  study  will  be  done  for  the  UH-60M. 
This  effort  aligns  with  the  use  case  from  the  CATE  work  and  will  add  a  more  thorough  O&S  analysis. 
To  accomplish  this  a  few  issues  must  first  be  addressed:  component  specific  susceptibility  and 
vulnerability  require  some  knowledge  of  the  component's  exposure  and  location  within  the  vehicle, 
assumptions  can  be  made  for  each  threat's  kill-chain  associated  with  Ph  but  data  is  needed  to  verify 
the  validity  of  those  assumptions,  and  the  best  distribution  for  representing  the  chance  of  threat 
occurrence  must  be  determined. 
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2.5  Develop  Case  Study  with  CATE  around  the  UH-60  Blackhawk 

This  section  details  the  approach  to  use  the  CATE  environment  on  a  specific  vehicle,  the  UH-60.  First  a 
new  method  for  calibrating  the  UH-60  vehicle  has  been  used.  From  the  calibrated  vehicle,  technologies 
were  implemented  to  represent  the  upgrades  and  to  investigate  possible  future  configurations. 

2.5.1  Calibration 

The  NDARC  files  have  been  calibrated  using  a  new  procedure.  The  Figure  38and  Figure  39  illustrate  the 
difference  between  the  previous  year's  calibration  method  and  the  updated  calibration.  Some  additional 
details  on  the  previous  year's  approach  can  be  found  in  the  previous  yearly  reports.  The  objective  is  to 
get  the  correct  NDARC  files  describing  the  RPTEM  engine  parameters,  aircraft  weight  calibration  factors, 
rotor  induced  and  profile  power  and  airframe  model.  The  derived  NDARC  files  will  be  used  as  the  vehicle 
baseline  used  in  the  other  parts  of  the  tool.  The  calibration  process  is  made  in  a  three  step  process,  going 
from  one  loop  to  another,  starting  inward:  calibrate  geometry,  calibrate  the  power  required  and  fuel  flow 
and  calibrate  the  weights.  Calibrated  geometry  gets  flat  plate  drag  and  layout  corrected,  which  will 
influence  power  required.  Power  required  influences  fuel  flow,  and  should  be  calibrated  in  that  order. 
However,  power  available  is  independent,  and  can  be  calibrated  on  its  own.  Both  can  influence  weight, 
so  component  weights  should  be  calibrated  last.  The  data  for  the  UH-60A  model  is  taken  from  a  variety 
of  sources.  The  NDARC  script  is  modified  from  a  SMR  example  packaged  with  the  NDARC  user  training 
files.  Geometry  for  the  UH-60L  is  derived  using  dimensions  from  the  UH-60A  math  model  [10]  and  the  UH- 
60A/L  operator's  manual.  There  are  no  external  differences  between  the  UH-60A  and  UH-60L,  so  it  is 
assumed  that  the  UH-60A  math  model  dimensions  apply. 

Empty  weight  is  calibrated  to  the  weight  information  for  a  manufactured  UH-60L  from  a  Sikorsky  weight 
statement  for  the  1571st  production  helicopter  using  technology  factors  [11],  Power  required  and 
available  data  and  fuel  flow  data  are  derived  from  the  UH-60A/L  Operator's  Manual.  The  NDARC 
calculations  of  power  required  are  calibrated  to  the  operator's  manual  data  using  a  two-step  process.  This 
calibration  is  more  complex  than  calibrating  the  geometry  and  empty  weight  due  to  the  large  number  of 
parameters  (around  60)  that  could  be  used  to  modify  power  required  estimations.  Thus,  the  first  step  is 
to  reduce  the  amount  of  variables  using  statistical  variable  screening  to  identify  which  parameters 
contribute  the  most  to  the  variability  of  power  required  in  hover  and  in  forward  flight.  The  second  step 
uses  an  optimization  algorithm  to  find  settings  for  the  parameters  identified  in  step  one  that  most 
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accurately  represent  the  performance  data  from  the  operator's  manual.  Any  settings  identified  as  not 
significant  to  the  variability  of  power  required  are  defaulted  to  the  SMR  example  setting  packaged  in  the 
training  files.  The  multi-objective  genetic  algorithm  is  used  to  minimize  the  two  objectives  given  below  by 
varying  the  parameters  identified  in  step  one.  This  algorithm  is  chosen  because  it  handles  non-linear, 
discontinuous  computation  models  and  performs  multi-objective  optimization.  For  simplicity,  the 
following  two  objectives  are  used. 

1.  Minimization  of  Root  Mean  Squared  Error  (RMSE)  of  power  required  to  hover  for  gross  weights 
between  12,000  lb  and  21,000  lb,  and  at  sea  level  standard  (SLS)  and  4000  ft,  95  °F 

2.  Minimization  of  RMSE  of  power  required  in  forward  flight  for  set  of  forward  speeds  ranging  from 
0  to  155  kts  at  gross  weights  of  16,000  lb  and  18,000  lb,  and  at  SLS  and  4000  ft,  95  °F 

Calibration  points  are  gathered  by  digitizing  performance  charts  from  the  Operator's  Manual.  Figure  40  is 
a  sample  image  showing  where  points  were  taken  for  power  required  and  fuel  flow  data. 


Figure  38  Previous  years  calibration  method 


□  Calibrate 
Geometry 


Loop  1 


□  Calibrate  Fuel 
Flow 

□  Calibrate  Power 
Required 

Loop  2 
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Weight 

Figure  39  Updated  calibration  method 
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Figure  40.  Sample  Plot  Indicating  Pulled  Data  Points  for  Power  Required  and  Fuel  Flow 


2.5.2  NDARC  Engine  Upgrade  Approach:  Power  Available  with  and  without 
Calibration 

2. 5.2.1  Introduction 

In  order  to  better  understand  how  to  model  upgrades  using  technologies,  within  the  CATE  environment, 
the  UH-60A  to  UH-60L  upgrade  was  used.  The  UH-60L  maintains  the  same  geometric  features  as  the  UH- 
60A  but  includes  an  upgraded  engine  and  improved  high-durability  gearbox.  The  purpose  of  this  exercise 
is  to  focus  on  the  engine  upgrade,  modeling  the  increase  in  Power  Available  and  change  in  Empty  Weight. 
The  UH-60L  is  equipped  with  two  General  Electric  T700-GE-701C  turboshaft  engines,  whereas  the  UH-60A 
is  equipped  with  two  General  Electric  T700-GE-701  turboshaft  engines.  Table  17  below  compares  the  two 
engines. 
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Table  17:  UH-60A/L  Engine  Comparison 


UH-60A 

UH-60L 

Engine 

GE  T700-GE-701 

GE  T700-GE-701C 

Rated  Horsepower  (IRP) 

1,560  shp 

1,800  shp 

2. 5.2.2  Scenarios 

Two  scenarios  were  identified  as  ways  to  model  the  engine  upgrade  in  NDARC.  The  scenarios  laid  out  here 
are  different  from  the  full  calibration  procedure.  These  scenarios  assume  that  the  technology  (i.e.  the 
engine)  can  be  modeled  within  CATE  environment  using  technology  factors  rather  than  by  a  complete 
NDARC  Referred  Parameter  Turboshaft  Engine  Model  (RPTEM).  To  simplify  the  process,  only  Power 
Available  was  considered.  The  model's  calibration  figure  of  merit  is  the  Root  Mean  Square  of  Errors 
(RMSE),  which  is  calculated  as  the  difference  between  the  NDARC  prediction  and  data  published  in  the 
UH-60L  Operator's  Manual  for  Maximum  Continuous  Power  (MCP),  Intermediate  Rated  Power  (IRP),  and 
One  Engine  Inoperative  (OEI).  The  Power  Curves  give  Power  Available  values  for  various  flight  conditions 
(i.e.  velocity,  altitude,  temperature).  To  study  the  effects  of  the  engine  improvements,  a  calibrated  UH- 
60A  model  was  used  as  the  baseline.  The  scenarios  are  as  follows: 

1.  Change  a  few  NDARC  parameters  to  see  impacts  on  RMSE's 

2.  Replicate  full  calibration  process  but  with  fewer  NDARC  parameters 

Scenario  1 

This  scenario  involves  changing  certain  NDARC  parameters  based  on  known  information  about  the  engine 
in  order  to  see  the  impacts  on  the  RMSE's.  The  following  parameters  (listed  in  Table  18)  were  identified 
based  on  the  fact  that  they  would  be  known  about  an  engine  even  if  it  hasn't  been  developed  yet.  Table 
18  also  includes  both  the  UH-60A  and  UH-60L  values  for  these  parameters. 


Table  18:  Scenario  1  NDARC  Parameters 


NDARC  Parameter 

Description 

UH-60A 

UH-60L 

Peng 

SLS  Engine  Power 

1560 

1800 

Plimit_es 

Engine  Shaft  Power  Limit 

2828 

3400 

Plimit_ds 

Drive  System  Power  Limit 

2828 

3400 
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Utilizing  ModelCenter,  the  UH-60A  aircraft  file  was  changed  to  include  the  UH-60L  parameters  from  Table 
18  without  changing  any  other  variables  and  the  RMSE's  were  calculated.  Table  19  summarizes  the  RMSE 
results  for  Scenario  1. 


Table  19:  RMSE  Results  for  Scenario  1 


Engine  Operating  Condition 

RMSE  (shp) 

MCP 

93.191 

IRP 

84.596 

OEI 

64.653 

It  is  important  to  note  that  the  errors  in  Table  19  remain  fairly  small  compared  to  the  expected  shaft 
horsepower  (~1800).  Figure  41  illustrates  an  example  of  the  error  in  MCP  for  each  atmospheric  operating 
condition  for  Scenario  1. 


Figure  41.  MCP  Data  vs.  MCP  RMSE  for  Scenario  1 

Each  point  in  Figure  41  compares  the  power  available  RMSE  for  each  different  flight  condition  (i.e.  altitude, 
velocity,  and  temperature)  to  the  actual  power  available  for  that  flight  condition  given  by  the  operator's 
manual.  This  figure  shows  that  for  many  different  flight  conditions  the  errors  are  generally  small  compared 
to  the  actual  power  available. 

Scenario  2 

This  scenario  seeks  to  replicate  the  full  calibration  process  but  by  only  varying  a  smaller  number  of  NDARC 
parameters.  Within  NDARC,  Power  Available  is  modeled  with  polynomial  expressions.  The  full  calibration 
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process  varies  all  the  coefficients  of  these  empirical  equations  in  order  to  minimize  the  RMSE  for  MCP, 
IRP,  and  OEI  power  available.  However,  for  this  scenario  this  is  not  the  case.  The  parameters  used  are 
shown  in  Table  20.  It  is  important  to  note  that  they  were  chosen  due  to  the  fact  that  they  are  important 
engine  parameters  that  could  be  used  to  model  simple  technology  improvements  with.  The  parameter 
values  used  in  Scenario  1  were  used  here  as  well. 


Table  20:  NDARC  Parameters  to  be  Varied  for  Scenario  2 


NDARC  Parameter 

Parameter  Description 

Nspec_tech 

Specification  Turbine  Speed 

ENG_SP0C_tech 

Specific  Power  at  MCP 

fPloss_xmsn 

Gear  Box  Loss 

eta_d 

Engine  Inlet  Efficiency 

fPlossJnlet 

Engine  Inlet  Loss 

fPloss_exh 

Engine  Exhaust  Loss 

After  varying  these  parameters  between  certain  ranges,  it  was  seen  that  this  scenario  resulted 
unsuccessful.  There  weren't  any  improvements  to  the  MCP,  IRP,  or  OEI  RMSE's. 

2. 5.2.3  Conclusions  and  Lessons  Learned 

Taking  all  of  these  scenarios  into  consideration,  one  can  conclude  that  Scenario  1  best  models  the  UH- 
60Ato  UH-60L  engine  upgrade  with  regards  to  power  available.  It  uses  actual  engine  information  that  will 
more  than  likely  be  known  to  the  decision  maker  rather  than  guestimates  and  doesn't  include  parameters 
that  will  require  "tweaking"  in  order  to  match  to  actual  engine  data  that  may  not  be  available. 

Throughout  this  process,  researchers  identified  a  number  of  lessons  that  have  been  learned.  First  and 
foremost,  there  is  a  need  for  a  way  to  improve  engine  modeling  within  the  CATE  environment.  These 
scenarios  rely  heavily  on  the  power  available  regressions  within  NDARC  that  are  used  to  model  the  engine 
that  are  fitted  to  power  curves  from  the  operator's  manual.  This  makes  evaluating  technologies  difficult 
without  having  the  power  curves  readily  available.  Also,  currently  in  the  CATE  environment,  the  sizing 
surrogate  engine  parameters  are  limited  to  technology  factors  on  Accessory  Power,  SFC,  and  Engine 
Weight  alone. 
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With  these  lessons,  researchers  have  been  able  to  suggest  a  few  possible  solutions.  The  first  solution 
would  be  to  utilize  higher  fidelity  engine  modeling  with  tools  such  as  NPSS.  This  would  allow  a  decision 
maker  to  tweak  parameters  related  to  specific  engine  technologies  within  the  engine  rather  than  the 
engine  as  a  whole.  This  will  still  require  curve  fitting  the  results  of  the  higher  fidelity  model  to  NDARC 
parameters,  similar  to  what  is  being  performed  in  the  NDARC  rotor  spreadsheet.  Another  possible  solution 
would  be  to  explore  other  engine  modeling  methods  within  NDARC.  After  some  investigation  into  how 
NDARC  can  model  an  engine,  it  was  discovered  that  the  turboshaft  engine  can  be  modeled  with  a  table 
lookup  rather  than  the  Referred  Parameter  Turboshaft  Engine  Model.  Though  this  may  be  a  very  good 
approach  for  existing  vehicles  where  there  is  a  wealth  of  data,  this  would  not  be  a  trivial  approach  for 
forecasting  future  engines  or  engine  technologies. 

2.5.3  UH-60M  Upgrade  and  Future  Technologies 

The  following  section  details  the  modeling  of  the  UH-60M  and  the  ITEP  engine  within  CATE.  CATE 
modeling  environment  is  based  on  a  sizing  task:  given  some  sizing  condition  and  mission  parameters, 
some  design  parameters  and  given  some  technology  calibration  factors  affecting  weight,  drag  and  engine 
performance,  a  vehicle  is  sized  and  the  characteristics  are  output.  The  CATE  environment  is  an  Excel-based 
implementation  of  regressions  of  the  sizing  task.  Table  21  shows  the  sizing  input  and  output  of  CATE 
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Table  21  CATE  input  and  output  variables 
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Among  the  technologies  affecting  the  performance  of  UH-60M,  it  has  an  upgrade  engine  (GE-710D 
turboshaft  engine)  and  Wide  Chord  Blades  (WCB).  In  the  previous  year,  the  UH-60M  has  been  modeled  in 
the  sizing  environment.  The  UH-60L  vehicle  parameters  were  changed  to  represent  the  technology: 

•  Increasing  VROC  and  maximum  speed  specifications  to  increase  the  power  of  the  installed  engine. 

•  Changing  the  engine  technology  factor  will  account  for  the  technology  on  the  T700-701D  that 
allowed  for  increased  power  without  increased  size. 

•  Changing  the  blade  loading  and  the  rotor  blade  factor  will  account  for  the  major  changes  of  the 
WCB. 

The  parameters  affected  by  the  upgrades  relative  to  the  UH-60L  are  illustrated  in  Table  22. 


Table  22  UH-60M  upgrades  modeling  assumptions,  compared  to  UH-60L 


Upgrade 

Projected  impact 

Wide  Chord  blade 

CWs  :  -9% 

TECH_blade  -12.1% 

701D  engine 

TECH_eng  -3.6% 

It  was  noted  that  the  increases  in  size  due  to  the  significant  increases  in  payload  and  crew  weight  causing 
the  vehicle  to  become  larger  overall  due  to  sizing  analysis.  Consequently,  an  alternate  approach  is 
proposed  in  the  following  section.  The  approach  uses  an  optimization  routine  to  converge  on  the  design 
inputs  that  lead  to  the  predicted  design  output.  In  other  words,  the  objective  is  to  output  the  mission 
capabilities  linked  to  new  technologies  without  a  change  in  vehicle  size. 

For  a  change  in  engine  performance,  the  optimization  problems  is  posed  as: 

Minimize  : 

^(predicted  power  -  modeled  power)2  +  n2 (predicted  TOGW  -  modeled  TOGW)2 
where  'n's  are  scaling  factors  (typically  one  over  the  nominal  value  of  the  associated  design  variable) 
With  respect  to:  VFWD,  cruise  time  The  optimizer  was  the  GRG  nonlinear  internal  excel  Solver  and  the 
regressions  already  present  in  CATE  were  used  for  the  process. 

For  the  proposed  UH-60M,  analysis,  only  the  engine  is  upgraded,  to  keep  the  weight  bookkeeping  easier 
to  understand.  Note  that  it  was  noted  in  the  previous  year  that  the  WCB  had  limited  impact  on  the  vehicle 
performance. 

First,  the  engine  weight  calibration  is  changed  from  the  nominal  value  of  1.43  to  1.397  (reduction  of  3% 
in  engine  weight)  to  account  for  the  increase  in  power  without  the  increase  in  engine  weight.  Then  the 
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optimization  routine  is  performed:  the  cruise  time  and  forward  velocity  are  changed  to  return  the 
expected  engine  power  without  changing  the  vehicle  weight. 

The  results  are  illustrated  in  the  Table  23. 


Table  23.  Engine  upgrade  in  CATE  results:  change  in  forward  velocity 


Parameter 

UH-60L  (CATE) 

UH-60M  (only  engine)  CATE 

Vfwd(KTAS) 

148.5 

151.3 

Cruise  time  (min) 

83.9 

78.9 

Gross  weight  (lb) 

18,601 

18,602 

MCP  (hp) 

1851 

1906 

Rotor  radius  (ft) 

28.46 

28.46 

The  results  show  that  the  engine  upgrade  allows  the  vehicle  to  fly  faster  with  a  more  powerful  engine 
without  introducing  changes  in  vehicle  weight  and  size.  The  design  cruise  time  is  reduced  by  5  minutes 
due  to  the  fact  that  the  cruise  is  at  higher  speed.  This  technique  allows  to  isolate  specific  performance 
characteristics  that  are  affected  by  an  upgrade  without  a  complete  re-design  of  the  vehicle.  The  same 
analysis  can  be  performed  by  replacing  the  forward  velocity  by  a  change  in  VROC,  and  these  results  are 
shown  in  Table  24.  In  this  case,  the  design  vertical  rate  of  climb  is  much  higher,  as  expected,  as  a  result  of 
a  more  powerful  engine. 


Table  24.  Engine  upgrade  in  CATE  results:  change  in  VROC 


Parameter 

UH-60L  (CATE) 

UH-60M  (only  engine  upgraded)  CATE 

VROC 

416 

922 

Cruise  time  (min) 

83.9 

80.89 

Gross  weight  (lb) 

18,601 

18,602 

MCP  (hp) 

1851 

1906 

Rotor  radius 

28.46 

28.46 

The  same  process  was  applied  to  evaluate  the  impact  of  installing  the  ITEP  engine  on  the  UH-60. 
Unfortunately,  the  bounds  of  the  surrogate  models  on  the  VROC  and  VFWD  did  not  allow  to  have  the 
expected  power  (3000hp  per  engine).  However,  the  proposed  process  is  similar  to  the  upgraded  engine 
exposed  in  the  previous  paragraphs:  expected  technology  factors  on  specific  fuel  consumption  and  engine 
weight  calibration  factors  are  changed  in  the  sizing  tab.  After,  the  optimization  is  executed  on  the  mission 
parameters  in  order  to  match  both  the  expected  power  and  the  vehicle  weight. 
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2.6  Survey  of  T echnology  Forecasting  T echniques  for  Complex  Systems 

Complex  system  design  and  assessment  is  a  challenging  task  exasperated  by  the  need  to  forecast  nascent 
technology  in  system  evaluation.  Proper  technology  forecasting  technique  selection  will  assist  decision¬ 
makers  to  understand  the  risks  involved  in  the  integration  of  emerging  technology  into  existing  or  new 
complex  system  developments.  A  research  summary  found  in  the  Appendix  surveys  the  field  of  technology 
forecasting  using  both  previous  technology  forecasting  survey  results  and  text  mining  on  academic 
literature  to  identify  60  unique  technology  forecasting  methodologies  and  associated  variations.  The 
literature  for  the  technologies  is  reviewed  to  place  the  technique  into  a  family,  describe  whether  it  was 
quantitative  or  qualitative,  indicate  whether  it  could  be  used  for  explorative  or  normative  forecasting, 
rate  12  criteria,  and  characterize  the  expected  results  of  the  technique.  A  technology  forecasting 
taxonomy  is  created  from  these  results.  This  taxonomy  can  be  used  to  guide  the  designer  or  decision 
maker  to  select  the  most  appropriate  technique  based  on  the  purpose  of  a  forecasting  exercise,  the 
characteristics  of  the  technology  to  be  forecasted,  and  the  amount  of  effort  and  resources  that  can  be 
expended  for  the  exercise. 
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4  Appendix 


4.1  Calibration  Parameters 

The  variables  used  in  the  calibration  are  illustrated  in  the  following  table 


Category 

Parameter 

Definition 

Peng 

Weight 

TECH_blade 

blade  weight  technology  factor 

TECH_hub 

hub  and  hinge  weight  technology  factor 

TECHrsupt 

rotor  support  weight  technology  factor 

TECH_rfold 

blade  fold  weight  technology  factor 

TECHtr 

tail  rotor  weight  (group  weight)  technology  factor 

TECH_tail 

tail  weight  technology  factor 

TECH_body 

basic  body  weight  technoloy  factor 

TECHcrash 

body  crashworthiness  weight  technology  factor 

TECHLG 

basic  landing  gear  weight  technology  factor 

TECH_LGcrash 

crashworthiness  weight  technology  factor 

TECH_cowl 

engine  cowling  weight  technology  factor 

TECH_supt 

engine  support  structure  weight  technology  factor 

TECHair 

air  induction  system  weight  technology  factor 

TECH_eng 

engine  weight  technology  factor 

TECHexh 

exhaust  system  weight  technology  factor 

TECHacc 

engine  accessories  weight  technology  factor 

TECH_plumb 

fuel  system  plumbing  weight  technology  factor 

TECH_tank 

fuel  tank  weight  technology  factor 

TECH_gb 

gear  box  weight  technology  factor 

TECHrs 

rotor  shaft  weight  technology  factor 

TECHds 

drive  shaft  weight  technology  factor  (aka  transmission  drive) 

TECH_RWfc_b 

boosted  rotary  wing  flight  control  weight  technology  factor 

TECH_RWfc_mb 

control  boost  mechanisms  rotary  wing  flight  control  weight  technology  factor 

TECH_RWfc_nb 

non-boosted  rotary  wing  flight  control  weight  technology  factor 

TECH_FWfc_nb 

non-boosted  fixed  wing  flight  control  weight  technology  factor 

TECH_RWhyd 

rotary  wing  flight  control  hydraulics  weight  technology  factor 

TECH_Dlelect 

anti-icing  electrical  system  weight  technology  factor 

TECHDIsys 

anti-ice  system  weight  technology  factor 

Rotor 

Power 
Required 
(must 
define  for 

each 

rotor) 

tiploss 

tip  loss  factor  B  (lift  zero  from  BR  to  tip) 

Ki_hover 

hover  induced  velocity  factor  (ratio  to  momentum  theory  induced  velocity 

Ki_climb 

axial  climb  induced  velocity  factor 

Ki_prop 

axial  cruise  induced  velocity  factor  (propeller) 

Ki_edge 

edgewise  flight  induced  velocity  factor  (helicopter) 

Ki_min 

minimum  induced  velocity  factor 

Ki_max 

maximum  induced  velocity  factor 

CTs_hind 

induced  blade  loading  for  induced  velocity  variation  with  thrust  in  hover 

kh2 

coefficient  for  induced  velocity  variation  with  thrust  in  hover 

mu_edge 

advance  ratio  for  induced  velocity  variation  with  edgewise  velocity  for  Ki_edge 

CTs_Pind 

induced  blade  loading  for  induced  velocity  variation  with  thrust  in  axial  cruise 

kel 

linear  coefficient  for  induced  velocity  variation  with  edgewise  velocity 

ke3 

exponent  coefficient  for  induced  velocity  variation  with  edgewise  velocity 

Xe 

exponent  for  induced  velocity  variation  with  edgewise  velocity 

CTs_Dmin 

blade  loading  for  minimum  profile  drag 

dO_hel 

constant  for  drag  equation  in  hover/edgewise 

dO_prop 

constant  for  drag  equation  in  prop  (axial) 

d2_hel 

quadratic  term  for  drag  equation  in  hover/edgewise 

d2_prop 

quadratic  term  for  drag  equation  in  prop  (axial) 

CTs_sep 

blade  loading  for  separation  (changes  cdbasic) 

dsep 

factor  for  drag  increment  (multiply  by  difference  in  blade  loading  wrt  to  separation  blade  loading) 

Xsep 

exponent  for  difference  in  blade  loading  wrt  to  separation  blade  loading 

fstall 

constant  in  stall  drag  increment 

dstalll 

factor  in  stall  drag  increment 

dstall2 

factor  in  stall  drag  increment 

Xstalll 

exponent  in  stall  drag  increment 

Xstall2 

exponent  in  stall  drag  increment 

MddO 

drag  divergence  mach  number  at  zero-lift 

dml 

linear  coefficient  in  drag  increment 

dm2 

exponent  coefficient  in  drag  increment 

Xm 

exponent  in  drag  increment 

sfcOC_tech 

specific  fuel  consumption  at  MCP  technology  factor 

Fuel  Flow 

SPOC_tech 

specific  power  at  MCP  technology  factor 

KffqO 

constant  for  referred  fuel  flow  required  at  power  required 

Kffql 

constant  for  referred  fuel  flow  required  at  power  required 

fPlossJnlet 

engine  inlet  loss 

fPloss_exh 

engine  exhaust  loss 

fPloss_xmsn 

gear  box  loss  (fraction  total  component  power  required) 

eta_d 

engine  inlet  efficiency 

Nspec_tech 

KspaO 

piecewise  linear  Kspa  =  KspaO  +  Kspal*theta,  Kspa  is  static  lapse  rate 

KspaO 

KspaO 

KspaO 

piecewise  linear  Kspa  =  KspaO  +  Kspal*theta,  Kspa  is  static  lapse  rate 

KspaO 

KspaO 

KspaO 

piecewise  linear  Kspa  =  KspaO  +  Kspal*theta,  Kspa  is  static  lapse  rate 

KspaO 

KspaO 

Kspal 

piecewise  linear  Kspa  =  KspaO  +  Kspal*theta,  Kspa  is  static  lapse  rate 

Kspal 

Kspal 

Kspal 

piecewise  linear  Kspa  =  KspaO  +  Kspal*theta,  Kspa  is  static  lapse  rate 

Kspal 

Power 

Available 

Kspal 

Kspal 

piecewise  linear  Kspa  =  KspaO  +  Kspal*theta,  Kspa  is  static  lapse  rate 

Kspal 

Kspal 

KspaO 

piecewise  linear  Xspa  =  XspaO  +  Xspal*theta,  Xspa  is  inlet  ram  air  exponents 

KspaO 

KspaO 

KspaO 

piecewise  linear  Xspa  =  XspaO  +  Xspal*theta,  Xspa  is  inlet  ram  air  exponents 

KspaO 

KspaO 

KspaO 

piecewise  linear  Xspa  =  XspaO  +  Xspal*theta,  Xspa  is  inlet  ram  air  exponents 

KspaO 

KspaO 

Kspal 

piecewise  linear  Xspa  =  XspaO  +  Xspal*theta,  Xspa  is  inlet  ram  air  exponents 

Kspal 

Kspal 

Kspal 

piecewise  linear  Xspa  =  XspaO  +  Xspal*theta,  Xspa  is  inlet  ram  air  exponents 

Kspal 

Kspal 

Kspal 

piecewise  linear  Xspa  =  XspaO  +  Xspal*theta,  Xspa  is  inlet  ram  air  exponents 

Kspal 

Kspal 
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This  research  seeks  to  review  and  characterize  technology  forecasting  techniques  to  create  a  taxonomy  for  use  in 
complex  system  analysis.  Previous  research  with  the  Capability  Assessment  and  Tradeoff  Environment  (CATE) 
identified  a  need  for  improved  technology  analysis  methods.  Past  technology  assessments  for  CATE  relied  on  the 
expert  elicitation  for  technology  assessment,  which  resulted  in  poor  forecasts  due  to  various  factors  associated  with 
the  technique.  Understanding  technology  forecasting  options  and  their  correct  application  will  remedy  the  problems 
encountered  by  the  CATE  research  team  and  other  groups  involved  in  forecasting  studies. 

The  research  is  accomplished  in  three  steps.  First,  techniques  referenced  by  existing  literature  surveys  on  the 
subject  are  extracted  and  compiled  together.  Second,  a  text  mining  approach  is  demonstrated  for  screening  literature 
and  identifying  any  recently  developed  techniques.  Third,  the  techniques’  associated  literature  is  reviewed  to 
characterize  each  technique  based  on  criteria  relevant  to  complex  systems.  A  few  suggestions  are  given  for  technique 
selection  based  on  the  taxonomy  to  illustrate  its  value. 

There  are  three  major  outcomes  of  this  research  -  application  of  text  mining  for  extracting  new  developments  in 
technology  forecasting,  an  exhaustive  taxonomy  of  techniques  indicating  their  characteristics  and  outcomes,  and  a 
suggested  approach  for  selecting  a  combination  of  techniques  for  complex  system  technology  forecasting.  Fifty-eight 
techniques  are  extracted  from  literature  survey.  An  analysis  of  12,000  research  database  results  found  two  new 
methods  -  a  class  of  methods  using  artificial  intelligence,  and  a  unique  technique  called  Fuzzy  Cognitive  Maps.  The 
taxonomy  is  created  for  the  sixty  methods  by  classifying  them  based  on  family,  quantitative  or  qualitative,  explorative 
or  normative,  expected  results,  and  then  characterized  according  to  twelve  criteria  taken  from  literature.  The  criteria 
covers  aspects  of  the  techniques  ability  to  make  predictions  relevant  to  its  development  time  frame,  life  cycle, 
evolution,  social  and  economic  impacts,  and  improvements.  The  criteria  also  considers  the  ease  and  cost  of  applying 
the  technique.  A  technique  can  be  selected  from  the  taxonomy  by  first  considering  the  desired  result(s)  to  create  a 
subset) s)  of  techniques,  then  weighting  their  technology  on  criteria  related  to  the  given  technique  criteria,  and  then 
applying  a  multi-criteria  decision-making  method  to  make  the  final  selection) s). 

The  analysis  to  find  new  techniques  is  similar  to  the  bibliometric  forecasting  method  and  results  in  a  few  lessons 
learned.  Simply  performing  factor  map  and  principal  component  decomposition  on  the  cleaned  natural  language 
processing  results  is  not  likely  to  return  any  significant  phrases.  Of  the  nodes  returned  in  this  analysis,  most  are  placed 
into  an  “other”  node.  A  keywords  and  key  phrases  list  to  represent  the  "language”  of  the  field  is  necessary  to  extract 
useful  information  from  the  databases.  Second,  the  results  of  text  mining  will  need  to  be  scrutinized  to  ensure  they  are 
relevant  to  the  information  desired.  These  lessons  indicate  that  the  process  is  more  hands-on  than  some  texts  indicate. 

The  taxonomy  can  be  improved  by  eliciting  ratings  from  technology  forecasting  experts.  The  limited  time  frame 
of  this  research  made  interaction  with  experts  infeasible  as  contacting  them,  creating  questionnaires,  receiving 
feedback,  evaluating  feedback,  and  iterating  until  consensus  is  a  lengthy  process.  Future  work  in  this  field  should 
refine  and  formalize  the  selection  methodology  by  applying  it  to  several  different  complex  system  technologies.  Given 
the  existing  selection  methods,  which  give  single  techniques,  the  method  should  instead  focus  on  forecasting  a 
complete  ecosystem  for  each  technology  in  a  simplified  manner.  Additionally,  researchers  will  benefit  by  working 
with  technology  forecasting  experts  to  refine  the  dimensions  for  complex  systems  and  match  these  dimensions  with 
the  expected  results  and  family  categorization  of  the  forecasting  techniques.  These  improvements  will  increase  the 
efficiency  and  repeatability  of  technology  forecasting  studies  for  complex  systems. 

This  study  demonstrates  the  use  of  text  mining  for  consuming  a  large  body  of  literature  for  new  technology 
forecasting  techniques.  However,  this  approach  can  be  applied  to  any  field  where  researchers  want  to  identify  the 
frontiers  of  its  research.  Finally,  the  forecasting  technique  taxonomy  created  through  this  research  provides  a  valuable 
resource  for  awareness  and  application  of  technology  forecasting  techniques. 
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Complex  system  design  and  assessment  is  a  challenging  task  exasperated  by  the  need  to 
forecast  nascent  technology  in  system  evaluation.  Proper  technology  forecasting  technique 
selection  will  assist  decision-makers  to  understand  the  risks  involved  in  the  integration  of 
emerging  technology  into  existing  or  new  complex  system  developments.  This  research  surveys 
the  field  of  technology  forecasting  using  both  previous  technology  forecasting  survey  results 
and  text  mining  on  academic  literature  to  identify  60  unique  technology  forecasting 
methodologies  and  associated  variations.  Then,  the  literature  for  the  technologies  is  reviewed 
to  place  the  technique  into  a  family,  describe  whether  it  was  quantitative  or  qualitative, 
indicate  whether  it  could  be  used  for  explorative  or  normative  forecasting,  rate  12  criteria, 
and  characterize  the  expected  results  of  the  technique.  A  technology  forecasting  taxonomy  is 
created  from  these  results.  This  taxonomy  can  be  used  to  guide  the  designer  or  decision  maker 
to  select  the  most  appropriate  technique  based  on  the  purpose  of  a  forecasting  exercise,  the 
characteristics  of  the  technology  to  be  forecasted,  and  the  amount  of  effort  and  resources  that 
can  be  expended  for  the  exercise. 


I.  Introduction 

IN  engineering,  technology  forecasting  is  concerned  with  the  desire  to  forecast  the  impact  a  technology  might  have 
on  system  performance.1  The  design  of  complex  systems  generally  relies  on  performance  estimates  for  decision¬ 
making.  Complex  systems  are  characterized  by  their  large  number  of  components  and  their  interactions  with  each 
other  and  the  environment.  Assessing  complex  system  performance  can  be  difficult  since  interactions  may  be  dynamic 
or  not  well  understood  and  because  the  whole  is  more  than  the  sum  of  the  parts.2  Assessing  nascent  technology 
integration  into  complex  system  designs  introduces  uncertainties  about  the  technology’s  impact  for  decision-makers 
to  manage  in  addition  to  uncertainties  in  system  performance  due  to  complexity.  Proper  selection  and  application  of 
technology  forecasting  techniques  will  give  decision-makers  an  improved  understanding  of  the  uncertainties  they  must 
manage,  as  well  as  multiple  aspects  to  consider  in  their  decision-making  process.  This  study  seeks  to  review  and 
characterize  applicable  technology  forecasting  techniques  for  use  in  complex  system  analysis  and  create  a  technology 
forecasting  taxonomy  for  reference  in  future  studies.  First,  the  problem  motivation  is  discussed,  followed  by  relevant 
background  information  on  technology  forecasting.  Then,  a  literature  review  on  the  state  of  technology  forecasting  in 
Aerospace  Engineering  and  any  relevant  technology  forecasting  surveys  is  given.  Next,  the  approach  and  results  for 
finding,  reviewing,  and  characterizing  technology  forecasting  is  discussed.  This  study  utilizes  text  mining  to  identify 
technology  forecasting  techniques  not  previously  mentioned  in  technology  forecasting  surveys.  Finally,  the  paper  is 
concluded  by  discussing  technique  selection  using  the  taxonomy  as  well  as  suggestions  for  forecasting  on  multiple 
dimensions. 


II.  Motivation 

Previous  research  with  the  Capability  Assessment  and  Tradeoff  Environment  (CATE)  identified  a  need  for 
improved  technology  analysis  methods.3  CATE  uses  “k-factor”  technology  representation  to  estimate  performance 
impacts,  which  allows  for  quantitative  representations  by  estimating  impacts  as  changes  with  respect  to  variables’ 
baseline  values.4,5  Past  technology  assessments  for  CATE  relied  on  the  Delphi  method,  or  expert  elicitation,  to  estimate 
technology  k-factors.  In  using  the  Delphi  method,  researchers  faced  three  difficulties.  First,  researchers  had  to  educate 
Subject  Matter  Experts  (SMEs)  on  this  representation  method,  which  is  a  departure  from  how  they  viewed 
technologies.  In  researchers’  experiences,  SMEs  are  more  concerned  with  how  to  get  a  technology  functioning  rather 
than  how  it  might  improve  or  change  a  system.  Second,  locating  and  contacting  an  appropriate  number  of  SMEs  was 
challenging  given  how  few  they  might  be  for  a  given  field.  Finally,  SMEs  were  prone  to  give  biased  estimates  because 
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they  need  to  “sell”  their  technology.  These  factors  lead  to  poor  technology  forecasts,  which  diminish  their  value  for 
decision-makers.3 

Understanding  technology  forecasting  options  and  correct  application  of  the  techniques  will  remedy  the  problems 
encountered  by  the  CATE  research  team  and  other  groups  involved  in  forecasting  work.  As  a  first  step  in  creating 
valuable  forecasts,  the  identification  of  relevant  forecasting  techniques  for  complex  systems  will  enable  future 
technology  researchers  to  easily  select  and  apply  techniques  to  their  specific  studies. 

III.  Background 

Technology  forecasting  can  be  broadly  characterized  as  exploratory  or  normative.  Exploratory  forecasting 
techniques  use  historical  trends  to  extrapolate  into  the  future  to  predict  what  might  happen.  Normative  forecasting 
techniques  start  with  future  goals  and  attempt  to  identify  necessary  levels  of  technological  improvement. 6  There  are 
four  general  techniques  that  exploratory  and  normative  forecasting  methodologies  use:  judgmental  or  intuitive 
methods,  extrapolation  and  trend  analysis,  models,  and  scenarios  and  simulations.  Judgmental  methods  use  opinion- 
based  forecasts.  These  can  be  disproportionately  biased  by  dominant  participants.  An  example  of  a  judgmental  method 
is  the  previously  mentioned  Delphi  method.  The  Delphi  method  is  a  structured  methodology  using  questionnaires  and 
feedback  to  elicit  expert  opinion  to  estimate  technology  impacts.  This  is  the  preferred  method  when  insufficient 
information  exists.  Extrapolation  and  trend  analysis  use  historical  data  for  making  forecasts.  S-curves,  or  growth 
curves,  are  an  example  of  this  methodology  type.  S-curves  assume  a  functional  form  of  a  previous  or  existing 
technology  growth  pattern.  The  drawback  of  this  method  is  the  large  amount  of  information  necessary.  Models  assume 
that  information  is  available  to  construct  and  solve  the  model  that  leads  to  a  forecast  at  some  time  in  the  future.  Finally, 
scenarios  and  simulations  assume  a  future  status  of  the  world  and  its  influence  on  a  technology  to  shape  the 
development  curve.7 

A  committee  assembled  to  create  a  persistent  disruptive  technology  forecast  methodology  suggest  using  many 
techniques  to  generate  forecasts  in  order  to  avoid  creating  poor  forecasts,  even  with  credible  data.7  Joseph  Martino 
and  John  Vanston  both  give  discussions  on  ranges  of  techniques  to  use.  Martino  suggested  a  broad  set  of  dimensions 
to  consider:  technological,  economic,  managerial,  political,  social,  cultural,  intellectual,  religious,  and  ecological.8 
Vanston  suggests  a  concise  set  of  five  views:  the  future  as  a  logical  extension  of  the  past,  an  intuitive  view  based  on 
experts,  pattern  analysis,  goal  analysis,  and  counter  puncher.9  Other  researchers  have  also  suggested  taking  a  multi¬ 
faceted  approach  to  ensure  that  the  ecosystem  surrounding  emerging  technologies  is  fully  considered. 10 

IV.  Literature  Review 

There  are  two  main  areas  to  review  literature.  The  first  is  in  Aerospace  Engineering,  where  a  significant  amount 
of  work  is  done  in  designing  complex  systems.  The  second  area  is  to  look  for  previous  surveys  of  technology 
forecasting  techniques. 

A.  Technology  Forecasting  Techniques  in  Aerospace  Engineering 

In  Aerospace  Engineering,  there  are  two  main  areas  related  to  technology  forecasting.  Researchers  either 
conducted  a  technology-specific  assessment1  U2,13  or  estimated  the  impacts  of  many  different  technologies  for  various 
complex  system  applications.14,151617 18  The  first  approach  gives  sub-system  and  component  level  predictions.  The 
second  approach  propagates  forecasts  at  lower-levels  to  system-level,  or  overall,  impacts.  This  approach  is  generally 
for  managing  technology  portfolios.  CATE  is  one  of  these  tools. 

The  general  methodology  for  creating  the  technology  impact  environments  is  based  on  Technology  Impact 
Forecasting  (TIF)4, 19,20  and  Technology  Identification,  Evaluation,  and  Selection  (TIES).21  TIF  aims  to  assess  the 
required  capabilities  to  meet  system  performance  objectives.4,19,20  TIES  is  an  approach  to  assess  specific  technologies’ 
impact  on  system  performance.  Additionally,  TIES  links  technology  readiness  levels  with  uncertainty  in  impact 
estimates  and  then  uses  probabilistic  tools  to  assess  uncertainty  in  future  technologies. 21 

A  few  other  studies  have  looked  at  implementing  new  techniques  to  assess  technology  k-factors.  A  recent  study 
briefly  discusses  expanding  technology  assessment  methods,  specifically  data  mining  and  mathematical  models  built 
on  category  theory,  graph  theory,  and  formal  concept  analysis.22  In  an  older  study,  multi-dimensional  growth  models 
for  technology  attributes  are  developed,  which  aids  in  complex  system  evaluation.  This  methodology  relies  on 
estimating  the  upper  physical  limits  for  all  attributes  and  thus  provides  a  framework  for  evaluating  the  technological 
limits  of  a  system  for  a  given  set  of  technologies.23  Another  study  combined  T  echnology  Synergy  Methodology,  which 
captures  2nd  and  higher  order  interaction  effects  between  technologies,  with  Dempster-Shafer  theory  in  order  to 
quantify  epistemic  uncertainty.  This  methodology  allows  for  multiple,  potentially  conflicting,  SME  beliefs  to  be 
aggregated  and  provide  a  better  understanding  of  the  uncertainty.24  In  this  author’s  experience,  both  of  these  studies 


2 

Aerospace  System  Design  Laboratory 
School  of  Aerospace  Engineering,  Georgia  Institute  of  Technology 


AE8900  MAV 


SPRING  2016 


Andrew  Smith 


have  not  been  widely  adopted  in  the  TIF  and  TIES  environment  creation  projects  likely  due  to  the  complexity  of  their 
processes.  Instead,  these  TIF/TIES  environments  tend  to  rely  on  the  Delphi  method  or  the  technology  assessments 
from  other  sources.  The  current  TIF/TIES  environments  also  lack  the  multi-faceted  technology  forecast  approach 
suggested  by  Martino  and  Vanston. 

B.  Previous  Surveys  of  Technology  Forecasting  Techniques 

Several  survey  studies  of  the  technology  forecasting  field  have  been  conducted.  Most  recently  in  2013,  Cho  and 
Daim  provided  a  fairly  comprehensive  survey  of  technology  forecasting  methods  and  their  origins  up  to  the  date  of 
the  research.25  Additionally  in  2013,  Kang,  Jang,  Lee,  and  No  investigated  developments  and  patterns  of  technology 
research  over  time  as  well  as  matching  industries  and  methods.26  In  2010,  The  National  Research  Council  Committee 
on  Forecasting  Future  Disruptive  Technologies  reviewed  the  high  level  forecasting  techniques  and  highlighted  a  few 
recent  advances.7  In  2008,  Firat,  Woon,  and  Madnick  summarized  technology  forecasting  techniques  and  applications. 
They  specifically  answered  questions  about  strengths  and  weaknesses  of  techniques.  The  researchers  reviewed  popular 
techniques  in  9  major  families:  expert  opinion,  trend  analysis,  monitoring  and  intelligence,  modeling  and  simulation, 
scenarios,  statistical,  descriptive,  creativity,  and  valuing  /  decision  /  economics  methods.  As  a  result,  the  report 
provides  discussions  on  overlapping  forms  of  forecasting  technology  developments  and  impacts.27  In  2004,  the 
Technology  Futures  Analysis  Methods  Working  Group  surveyed  several  different  fields  where  technology 
forecasting,  or  assessment,  was  occurring.  They  synthesized  the  various  methods  employed  into  a  table,  which 
indicates  the  method’s  family,  the  qualitative  or  quantitative  nature  of  the  data  used  in  the  method,  and  if  the  technique 
is  explorative  or  normative.28  In  2003,  Martino  summarized  recent  advanced  in  both  methodology  improvement  and 
novel  methodologies.29  In  2001,  Slocum  and  Lundberg  reviewed  families  of  forecasting  methods  with  a  focus  on  TRIZ 
methods.30  In  1998,  Meade  and  Islam  reviewed  technology  forecasting  selection  and  model  combinations,  resulting 
in  some  guidelines  for  using  multiple  models.  However,  their  research  focused  on  diffusion  models.31  Finally,  in  1991, 
Porter,  Roper,  Mason,  Rossini,  and  Banks  reviewed  methods  by  the  parameters  being  forecasted.32 

There  have  also  been  several  studies  to  investigate  methodologies  for  technique  selection.  In  2013,  Intepe,  Bozdag, 
and  Koc  created  a  method  using  fuzzy  technique  for  order  preference  by  similarity  to  ideal  solution  (TOPSIS)  to  find 
the  most  appropriate  technique  as  evaluated  on  7  selection  criteria.33  In  2008,  Cheng,  Chen,  and  Chen  performed  a 
similar  study  with  8  criteria  and  a  fuzzy  analytic  hierarchy  process  (AHP)  for  method  selection.34  In  2002,  Mishra, 
Deshmukh,  and  Vrat  provided  a  methodology  for  matching  forecasting  technique  and  technology  on  22 
characteristics.35 

As  to  this  author’s  knowledge,  there  are  no  studies  which  indicate  systematic  ways  to  combine  techniques  to  create 
the  complete  forecasting  ecosystem.  The  studies  which  create  methodologies  for  selecting  a  forecasting  technique  did 
not  apply  the  process  to  all  available  techniques,  nor  did  surveys  attempt  to  characterize  expect  results  from  the 
technique.  This  study  intends  to  re-survey  the  technology  forecasting  field  to  find  any  recent  developments,  and  then 
characterize  each  technique  based  on  criteria  relevant  to  complex  systems  to  create  a  technology  forecasting 
taxonomy,  which  includes  the  type  of  expected  results.  Finally,  using  the  expected  results,  the  research  will  suggest 
combinations  of  techniques  which  result  in  a  well-rounded  forecast. 

V.  Approach 

A  review  of  technology  forecasting  techniques  is  conducted  in  three  steps  with  the  end  goal  of  creating  a 
technology  forecasting  taxonomy  for  complex  systems. 

1.  Technology  forecasting  techniques  and  methods  are  extracted  from  the  previously  discussed  technology 
forecasting  surveys. 

2.  Databases  are  queried  for  technology  forecasting  methods  and  the  results  combined  and  analyzed  for  new 
techniques. 

3.  Relevant  papers  for  the  identified  techniques  are  identified  and  reviewed  to  characterize  each  technique  based 
on  the  criteria  relevant  for  complex  systems.  Additionally,  expected  results  of  the  forecasting  technique  are 
categorized. 

4.  The  characterization  and  result  categorization  are  assembled  into  a  technology  forecasting  taxonomy  to  help 
future  researchers  efficiently  select  techniques. 

Finally,  some  suggestions  are  given  for  technique  selection  based  on  the  taxonomy  and  literature  for  considering 
multiple  dimensions. 

In  step  2,  records  are  taken  from  the  ProQuest  Research  Library,  EBSCO  Academic  Search  Complete,  and  Web 
of  Science.  These  databases  are  selected  due  to  their  breadth  of  topics.  They  are  also  basic  research  databases,  which 
are  assumed  to  contain  any  recent  advances  in  technology  forecasting.  The  search  terms  used  are  technology’  NEAR/2 
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forecast *  technology’  NEAR/2  trend,  technology’  NEAR/2  trajectory,  technology’  NEAR/2  foresight,  tech*  NEAR/2 
intelligence,  forecast*  NEAR/2  technique,  and  forecast*  NEAR/2  methodology.  The  phrase  NEAR/2  activates 
proximity  searches  where  results  with  the  terms  before  and  after  NEAR/2  are  within  2  words  of  each  other.  The  search 
item  tech  *  NEAR/2  intelligence  is  used  to  find  results  from  industry  methods  that  use  the  terminology  “technology 
intelligence”  and  “competitive  technical  intelligence”,  so  these  terms  are  included  to  capture  any  relevant  articles.27 
Any  articles  related  to  meteorology  or  demand  forecasting  are  removed.  VantagePoint  text-mining  software  is  used  to 
remove  duplicate  records  and  to  rapidly  consume  abstract  information.  VantagePoint  provides  tools  to  clean  text  with 
a  thesaurus  and  then  use  Natural  Language  Processing  (NLP)  to  extract  words  and  phrases.36  The  most  challenging 
aspect  of  this  work  is  analyzing  the  large  body  of  literature  on  the  subject  of  technology  forecasting.  VantagePoint 
utilizes  text  mining  to  greatly  enhance  assessing  literature.  To  find  new  forecasting  methods,  a  key  word  list  is 
generated  from  the  extracted  techniques  from  the  surveys.  Then,  VantagePoint  is  used  to  pull  these  key  words  from 
the  abstracts.  Next,  VantagePoint’ s  NLP  algorithm  is  used  to  extract  general  phrases  from  the  abstracts.  A  co¬ 
occurrence  matrix  is  used  with  the  key  word  list  and  the  NLP  phrases  to  find  where  the  key  words  do  not  occur  with 
the  NLP  phrases  as  they  may  be  new  techniques.  These  “new  technique  phrases”  are  hand-reviewed  to  remove  any 
phrases  that  are  known  to  not  be  related  to  a  technique.  Then,  the  “new  technique  phrases”  are  analyzed  using  both  a 
factor  map  and  principal  component  decomposition  (PCD).  The  factor  map  analysis  uses  small-increment  Kaiser 
Varimax  Rotation  to  cluster  phrases  together  that  capture  the  most  records,  while  the  PCD  analysis  creates  co¬ 
occurrence -based  principal  components  to  cluster  phrases  together  to  capture  the  most  records.37  Both  analyses  will 
filter  the  results  into  areas  of  technology  forecasting  research,  which  are  then  be  explored  to  identify  if  it  is  a  new 
technique  or  not. 

In  step  3,  the  general  techniques  identified  in  step  2  are  reviewed  to  classify  and  identify  characteristics  relevant 
to  complex  systems.  These  characteristics  are  a  subset  of  the  characteristics  mentioned  in  the  literature33,34,35  and  are 
given  in  Table  1. 

Table  1.  Technology  Forecasting  Technique  Criteria _ 


Criteria  Reference 


Capability  to  forecast  incremental  change 

35 

Capability  to  forecast  radical  innovations 

35 

Capability  to  forecast  modular  technologies 

35 

Life  cycle  prediction  capability 

35 

Capability  to  forecast  for  stipulated  time  horizon 

35 

Data  availability 

33, 

34, 

35 

Data  validity 

33, 

34, 

35 

Technology  development  predictability 

33, 

34, 

35 

Technology  similarity 

33, 

34, 

35 

Method  of  adaptability 

33, 

34, 

35 

Ease  of  technique  implementation 

33, 

34, 

35 

Cost  of  technique  implementation 

33, 

34, 

35 

Technology  forecasting  for  complex  systems  is  largely  concerned  with  how  the  particular  system’s  behavior  will 
change  and  not  necessarily  replacements  for  the  complex  system.  For  forecasting,  techniques  concerned  with  diffusion 
or  acceptance  of  technology  are  not  relevant.  The  criteria  selected  for  evaluating  techniques  ensures  that  the  character 
of  the  technology  change  is  considered  (incremental  change,  radical  change,  or  modular  behavior),  the  life  cycle  of 
the  technology,  the  time  frame  for  which  a  technology  is  being  considered,  what  is  known  about  the  technology 
currently  (data  availability  and  validity),  and  the  similarity  of  the  technology  to  previous  technologies.  Additionally, 
the  criteria  considers  how  the  technique  is  implemented,  as  users  are  not  likely  to  attempt  difficult  or  expensive 
methods  (ease  and  cost  of  technique  implementation).  After  the  techniques  are  characterized,  the  expected  results  are 
also  characterized  to  give  researchers  an  indication  of  what  to  use  the  technique  for.  The  characterizations  from  step 
3  are  considered  in  conjunction  with  literature  consensus  on  multiple  dimensional  forecasting.  This  work  will  provide 
future  researchers  a  quick  technique  reference  for  use  in  forecasting  exercises  with  complex  systems. 

VI.  Implementation  and  Analysis 

There  are  three  major  outcomes  of  this  research  -  a  list  of  possible  technology  forecasting  techniques,  a  taxonomy 
of  techniques  applicable  to  complex  systems  and  their  characteristics  and  outcomes,  and  a  discussion  on  selecting  a 
combination  of  techniques  for  complex  system  technology  forecasting. 
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A.  Technology  Forecasting  Techniques 

Two  hundred  individual  techniques  are  extracted  from  the  technology  forecasting  surveys.  The  table  given  by  the 
Technology  Futures  Analysis  Methods  Working  Group  is  used  as  a  starting  point  for  assembling  a  matrix  because  it 
categorizes  techniques  by  family  (Creativity,  Descriptive  and  Matrices,  Expert  Opinion,  Modeling  &  Simulation, 
Scenarios,  Statistical,  Trend,  and  Valuing/Decision/Economic),  quantitative  or  qualitative,  type  (Explorative  or 
Normative),  and  gives  references  for  further  information  about  the  technique.28  There  are  51  techniques  given  in  the 
table.  Comparing  these  techniques  with  the  other  surveys  indicated  that  the  methods  listed  by  the  Technology  Futures 
Analysis  Methods  Working  Group  consist  of  many  different  variations  (especially  for  trends,  where  Meade  and  Islam 
list  29  different  curves31).  The  techniques  from  the  other  surveys  are  reviewed  for  addition  to  the  table,  resulting  in 
adding  the  following  7  techniques:  Artificial  Neural  Networks  (ANN),  causal  layered  analysis,  collaborative  methods, 
heuristic  methods,  hybrid  models,  tech  sequence  analysis,  and  wild  cards. 

All  of  the  techniques  extracted  from  the  surveys  are  used  to  create  a  keyword  list  for  analyzing  research  databases. 
Approximately  12,000  papers  are  returned  from  the  research  database  queries.  The  search  results  are  uploaded  into 
VantagePoint  and  then  cleaned  to  remove  duplicates  based  on  author  and  paper  title.  Then,  the  keywords  are  input 
into  VantagePoint.  VantagePoint  searches  the  abstracts  to  find  and  pull  the  keywords  from  the  papers.  Then, 
VantagePoint’ s  NLP  algorithm  is  applied  to  the  abstracts  to  pull  phrases.  These  phrases  are  cleaned  twice  -  first  to 
change  any  British  spellings  to  American  (using  the  British  thesaurus)  and  then  cleaned  using  a  general  fuzzy 
algorithm,  which  truncates  words  to  their  root.  Next,  a  co-occurrence  matrix  analysis  is  created  to  see  where  the 
keywords  and  NLP  phrases 
occurred  together  and  where 
they  did  not.  The  matrix  is 
filtered  to  remove  results  with 
more  than  1  co-occurrence. 

Then,  the  remaining  NLP 
phrases  are  selected  and  used  to 
create  a  subgroup  within  the 
NLP  phrase  list.  This  group  is 
reviewed  by  hand  to  remove 
phrases  that  are  known  to  not  be 
techniques  (such  as  the 
publishing  company  name 
Elsevier).  This  sub-group  is  then 
analyzed  using  both  the  factor 
map  and  the  PCD  analyses 
available  in  VantagePoint. 

Figure  1  shows  the  Factor  Map 
for  potential  new  forecasting 
techniques.  The  lines  between 
the  nodes  indicate  that  the  terms 
are  related  by  being  included  in 
the  same  record(s).  The  map 
indicates  the  following  potential 
techniques: 

•  Bayesian  methods 

•  Genetic  programming 

•  Collaborative  foresight 

•  Real-time  forecasts 

•  Hybrid  approach 

•  Artificial  intelligence 

•  Fuzzy  methods 
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Figure  1.  Factor  Map  of  Potential  New  Forecasting  Techniques 
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The  other  phrases  (futures  methods  and  DESIRE  approach)  are  unrelated  phrases  to  forecasting  techniques.  Of  the 
potential  techniques  listed,  Bayesian  methods,  collaborative  foresight,  and  hybrid  approach  are  already  included  in 
the  surveys.  These  terms  were  not  a  part  of  the  key  word  list,  so  the  list  of  phrases  of  potentially  new  techniques 
included  them.  Additionally,  genetic  programming  is  considered  part  of  heuristic  methods  by  this  author.  The  term 
genetic  programming  was  also  not  included  in  the  key  words.  When  using  this  technique,  failing  to  include  all  of  the 
correct  terms  results  in  some  older  techniques  remaining  in  the  analysis.  The  remaining  techniques  to  review  for 

possibility  of  inclusion  in  the 
updated  technology  forecasting 
taxonomy  are  Bayesian 

methods,  real-time  forecasts, 

artificial  intelligence,  and 
fuzzy  methods. 

Before  reviewing  these 
phrases,  the  Principal 

Component  Decomposition 
analysis  is  used  in  an  attempt  to 
uncover  other  possible 

techniques  and  to  compare 
methods.  Figure  2  shows  the 
map  created  using  Principal 
Component  Decomposition. 
The  map  indicates  the 
following  potential  techniques: 
Combined  forecast 
Learning  process 
Participatory  approach 
Computational  intelligence 
Perfect  foresight 
Fuzzy  methods 
Machine  learning 
Artificial  intelligence 
Hybrid  approach 
Data  mining 
Volatility  forecast 
Episodic  foresight 
The  other  phrases 
(technology  intelligence  and 
management  system)  are 
unrelated  to  forecasting 
techniques.  The  key  word  list 
included  phrases  similar  to 
participatory,  combined 

forecast,  hybrid  approach,  and 
data  mining  (part  of 
bibliometrics),  but  not 
explicitly  these  words.  This 
again  reinforces  how  sensitive  this  technique  is  to  its  inputs.  The  remaining  techniques  are  learning  process, 
computational  intelligence,  perfect  foresight,  fuzzy  methods,  machine  learning,  artificial  intelligence,  volatility 
forecast,  and  episodic  foresight.  This  analysis  produced  many  more  results  than  the  factor  map.  However,  the  factor 
map  analysis  gave  two  results  that  the  PCD  did  not  (real-time  forecasts  and  Bayesian  methods).  Of  the  PCD  results, 
the  factor  map  only  gave  artificial  intelligence  and  fuzzy  methods.  The  difference  in  results  is  due  to  different  data 
reduction  techniques  used  by  factor  map  analysis  and  PCD. 

The  research  papers  associated  with  the  phrases  are  reviewed  to  check  that  they  are  actual  techniques  for 
forecasting.  Two  new  techniques  emerged:  artificial  intelligence38,39  and  the  fuzzy  cognitive  map.40  Machine  learning 
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is  generally  considered  subset  of  artificial  intelligence  in  computing  fields,  so  this  technique  is  considered  a  variation 
of  artificial  intelligence  methods. 

Both  factor  map  and  PCD  identified  artificial  intelligence  and  fuzzy  method  as  new  forecasting  techniques.  Factor 
map  is  more  efficient  as  it  represented  the  NLP  phrase  list  of  potential  new  techniques  with  13  phrases  compared  to 
PCD’s  18  phrases.  However,  PCD  ran  faster  and  could  handle  larger  data  sets  than  the  factor  map  analysis. 

The  text  mining  method  to  find  new  techniques  is  similar  to  bibliometric  forecasting,  so  the  insights  from  this 
study  are  useful  for  future  applications.  First,  performing  factor  map  or  PCD  analysis  on  the  cleaned  NLP  phrases 
may  not  return  any  significant  results.  Of  the  nodes  returned  in  this  study,  most  records  are  placed  into  an  “other” 
node,  which  is  useless  information.  A  keywords  and  key  phrases  list  representing  the  “language”  of  the  field  are 
necessary  to  extract  useful  information  from  the  database  records.  For  applying  bibliometric  techniques  to  complex 
systems,  describing  the  system  using  functional  words  will  be  invaluable  for  creating  the  key  word  list.  Second,  the 
results  of  text  mining  will  need  to  be  scrutinized  to  ensure  they  are  relevant  to  the  information  desired.  The  application 
of  text  mining  illustrates  its  power,  but  the  lessons  learned  indicate  that  the  method  may  be  less  straightforward  than 
some  literature  indicates. 

B.  The  Complex  System  Technology  Forecasting  Taxonomy 

After  collecting  the  techniques,  they  are  classified  based  on  family,  quantitative  or  qualitative,  and  explorative  or 
normative.  They  are  additionally  characterized  by  the  criteria  in  Table  1,  which  are  defined  as  follows: 

•  Capability  to  forecast  incremental  change  indicates  how  well  the  technique  can  handle  evolutionary 
technology  predictions,  such  as  derivative  designs.35 

•  Capability’  to  forecast  radical  innovations  indicates  how  well  the  technique  can  handle  revolutionary 
technology  predictions,  or  blank  sheet  designs. 35 

•  Capability’  to  forecast  modular  technologies  indicates  how  well  the  technique  can  handle  technologies  whose 
components  can  be  re-arranged  and/or  swapped  to  provide  different  functionality. 35 

•  Life  cycle  prediction  capability  indicates  how  well  the  technique  is  suited  to  all  aspects  of  a  technology’s  life, 
from  inception  to  obsolescence  of  the  technology. 35 

•  Capability’  to  forecast  for  stipulated  time  horizon  indicates  how  well  the  technique  is  suited  for  long-term 
forecasting.  Ratings  of  0  indicate  short-term  and  ratings  of  1  indicate  long-term. 35 

•  Data  availability’  is  the  quantity  of  data  required  to  use  the  technique.33 

•  Data  validity  is  how  well  the  data  should  correspond  to  the  metric  of  interest  for  the  technique.33 

•  Technology’  development  predictability  indicates  how  suited  the  technique  is  to  predicting  the  movement  of 
technology  from  basic  research  to  production.33 

•  Technology  similarity  indicates  how  similar  the  new  should  be  to  an  existing  technology  to  use  the  technique.33 

•  Method  of  adaptability  indicates  the  extent  to  which  the  technique  depends  on  expert  opinions  (higher  rating 
indicates  higher  reliability  on  experts).33 

•  Ease  of  technique  implementation  indicates  how  easy  it  is  to  grasp  and  apply  the  technique.33 

•  Cost  of  technique  implementation  indicates  expected  level  of  resource  commitment  to  apply  the  technique.33 

Additionally,  the  expected  results  are  characterized  using  the  following  definitions  based  on  literature  associated 
with  each  technique. 

•  Acceptability  indicates  that  the  technique  will  or  will  not  be  acceptable  to  a  society,  while  rate  is  used  to 
clarify  that  the  technique  will  give  an  indication  of  adoption. 

•  Alternatives  means  that  several  options  for  a  function  that  a  technology  performs  will  be  given  or  that  several 
scenarios  surrounding  the  technology  will  be  given. 

•  Any  indicates  that  the  technique  is  flexible  enough  to  be  used  to  generate  whatever  the  user  desires. 

•  Behavior  is  used  to  indicate  when  a  technique  helps  users  understand  linkages  between  actions  and  trends  and 
technology. 

•  Decision  indicates  that  the  technique  is  likely  to  prescribe  what  decision  to  make  based  on  value. 

•  Economic  trends  or  economic  value  indicates  that  the  technique  is  concerned  with  the  profitability  of  the 
technology. 

•  Impact  identification  indicates  that  the  technique  returns  knowledge  about  where  impacts  of  a  technology  can 
be  expected. 

•  Metric  estimates  or  metric  values  indicates  that  the  technique  returns  actual  values  for  metrics  for  which  the 
technology  is  being  measured. 
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•  Metrics  simply  indicates  that  the  technique  indicates  what  metrics  should  be  used  to  assessing  a  technology. 

•  Obsolescence  scenarios  indicates  that  the  technique  gives  situations  in  which  something  is  no  longer 
necessary. 

•  Probabilities  of  outcomes  indicates  that  the  technique  gives  users  an  idea  of  the  uncertainty  of  a  particular 
event  or  metric  value  coming  to  fruition. 

•  Roadmap  to  scenario  is  used  to  separate  a  technique  from  technology  development  when  the  technique  deals 
specifically  with  pathways  to  an  event  occurrence. 

•  Technology’  development  is  related  to  both  the  pathway  of  a  technology  from  basic  research  to  production  as 
well  as  the  timeline  of  the  technology’s  development. 

An  example  of  the  technology  taxonomy  is  given  in  Table  2  illustrating  the  characterizations  for  one  of  the  new 


techniques,  artificial  intelligence. 

Table  2.  Sample  Technology  Taxonomy 


Method  |variations] 

Artificial 

Intelligence 

[Machine 

Learning] 

Family 

Statistical 

Quant  (H)  or  Qual  (S) 

H/S 

Exploratory  or  Normative 

Ex 

Capability  to  forecast  incremental  change  |0  -  cannot,  1  -  can] 

1 

Capability  to  forecast  radical  innovations  |0  -  cannot,  1  -  can] 

0 

Capability  to  forecast  modular  technologies  |0  -  cannot,  1  -  can] 

0 

Life  cycle  prediction  capability  |0  -  cannot,  1  -  can] 

0 

Capability  to  forecast  for  stipulated  time  horizon  |0  -  short-term  only,  1  -  any  time  frame] 

0 

Data  availability  |0  -  none  or  little,  1  -  significant] 

1 

Data  validity  |0  -  weak,  1  -strong] 

1 

Technology  development  predictability  |0  -  cannot,  1  -  can] 

0 

Technology  similarity  |0  -  unsimilar,  1  -  similar] 

1 

Method  of  adaptability  |0  -  no  reliance  on  ExOp,  1  -  full  reliance  on  ExOp] 

0 

Ease  of  technique  implementation  |0  -  easy,  1  -  hard] 

1 

Cost  of  technique  implementation  |0  -  cheap,  1  -  expensive] 

1 

Expected  Results  for  Complex  Systems 

Metric  value 

References 

38,  39 

The  fully  technology  forecasting  technique  taxonomy  is  given  in  Table  4  in  the  Appendix.  Table  4  lists  60 
technology  forecasting  techniques  and  their  variations,  characteristics  for  each,  and  references  for  further  information 
on  the  technique.  This  table  is  useful  for  users  new  to  the  field  of  technology  forecasting  to  find  techniques  appropriate 
to  their  study.  The  characteristic  values  are  based  on  reviews  of  literature.  However,  the  taxonomy  would  be  improved 
by  eliciting  ratings  from  technology  forecasting  experts.  The  limited  time  frame  of  this  research  made  interaction  with 
experts  infeasible  as  contacting  experts,  creating  questionnaires,  receiving  feedback,  evaluating  feedback,  and  iterating 
until  consensus  is  a  lengthy  process. 

C.  Using  the  Taxonomy  for  Technique  Selection 

There  are  several  ways  to  use  the  technology  taxonomy  to  select  a  single  technique.  A  simple  method  is  described 
by  Mishra  and  Deshmukh  where  a  technology  is  rated  on  characteristics  that  align  with  the  forecasting  technique’s 
characteristics.  Next,  a  multi-criteria  decision  making  technique  is  used  to  evaluate  which  technique  the  technology’s 
score  is  closest  to.35  The  technique  described  by  Intepe,  Bozdag,  and  Koc  uses  expert  judgement  combined  with  fuzzy 
logic  to  rate  techniques  on  given  criteria  for  a  particular  area.34  Use  Table  3  based  on  Ref.  33-35  for  guidance  on  the 
technology  characteristics  that  correspond  with  forecasting  technique  characteristics. 
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Table  3.  Matching  Technology  Criteria  for  Technique  Criteria 


Technology  Criteria 

Technique  Criteria 

Evolutionary  change 

Capability  to  forecast  incremental  change 

Revolutionary  change 

Capability  to  forecast  radical  innovations 

Modularity  of  technology 

Capability  to  forecast  modular  technologies 

Life  cycle 

Life  cycle  prediction  capability 

Time  frame  of  interest 

Capability  to  forecast  for  stipulated  time  horizon 

Existing  data  availability 

Data  availability 

Exiting  data  validity 

Data  validity 

Technology  readiness  level 

Technology  development  predictability 

Existing  similar  technologies 

Technology  similarity 

Amount  of  existing  information 

Method  of  adaptability 

Time  available  for  study 

Ease  of  technique  implementation 

Resources  available  for  study 

Cost  of  technique  implementation 

With  the  above  table,  forecasters  can  rate  the  technology  for  each  criteria,  and  then  use  multi-criteria  decision  making 
techniques  such  as  technique  for  order  preference  by  similarity  to  ideal  solution  (TOPSIS)  to  find  techniques  that 
match.  Note  that  for  technology’  readiness  level  and  amount  of  existing  information  are  inverse  of  the  rating  of  the 
technology’  development  predictability  and  method  of  adaptability’.  A  high  technology  readiness  level  may  not  need  a 
technique  for  development  predictability.  A  large  body  of  work  regarding  a  technology  may  be  able  to  use  techniques 
that  do  not  rely  on  expert  opinions. 

Neither  of  the  previously  discussed  selection  techniques  consider  forecasting  for  the  entire  ecosystem  for  a 
technology.  Using  the  taxonomy  in  Table  4,  users  can  use  the  “Family”  and  “Expected  Results”  category  to  match 
technique  with  a  part  of  the  ecosystem.  For  example,  consider  the  5  varieties  given  by  Vanston:  the  future  as  a  logical 
extension  of  the  past,  an  intuitive  view  based  on  experts,  pattern  analysis,  goal  analysis,  and  counter  puncher.9  Based 
on  these  criteria,  users  should  select  one  technique  from  the  Trend  family  (which  gives  metric  values),  a  technique 
from  the  expert  opinion  (which  can  give  any  data  that  the  user  is  requesting),  a  technique  that  gives  insights  on 
behaviors,  a  normative  technique  that  gives  information  related  to  technology  development  or  alternatives,  and  lastly, 
a  creativity  technique  that  gives  alternatives.  While  these  5  forecasts  give  a  variety  of  useful  information  about  the 
technology  in  consideration,  there  are  still  other  areas  untouched,  such  as  the  life  cycle  of  the  technology,  or  social 
impacts.  Adding  these  areas  to  the  list  of  Vanston,  or  using  the  extensive  list  given  by  Martino,8  would  cover  more 
dimensions  but  also  increase  the  workload  of  the  technology  assessment. 

Not  all  complex  system  technology  forecasts  may  benefit  from  a  forecast  in  the  same,  broad  areas.  Instead  of  using 
all  areas,  technology  forecasters  and  experts  could  evaluate  the  likelihood  of  a  technology’s  economic,  managerial, 
political,  social,  cultural,  intellectual,  religious,  and  ecological  impact  (the  dimensions  given  by  Martino8).  Note  that 
these  dimensions  are  in  addition  to  the  performance  of  the  technology.  To  illustrate  this  idea,  consider  two  nominal 
cases.  Case  A  is  evaluating  a  new  rotor  technology  for  an  existing  single  main  rotor  system.  Case  B  is  the  introduction 
of  a  new  vertical  take-off  and  landing  (VTOL)  short-haul  vehicle  for  urban  commuting. 

In  Case  A,  the  new  rotor  technology,  forecasters  are  likely  to  be  most  concerned  with  how  it  will  impact  system 
performance.  In  addition,  they  should  be  interested  in  its  economic  impact,  managerial  or  development  track,  any 
political  difficulties  if  it  is  for  defense  systems,  and  possibly  ecological  impacts  such  as  noise.  However,  Martino’s 
other  dimensions  are  not  important  for  this  particular  technology.  Following  the  performance  impact,  the  technology’s 
economic  and  managerial  forecasts  will  be  most  important.  The  analyst  should  select  a  technique  that  gives  economic 
results  with  the  high  data  availability  and  data  validity  and  a  technology  development  result  technique  with  high  data 
availability  and  data  validity.  Political  and  ecological  issues  may  be  secondary  as  it  is  a  derivative  technology,  so  the 
analyst  should  choose  a  technique  from  the  expert  opinion  family  for  any  political  considerations  (a  defense  firm  may 
be  interested  in  likelihood  of  upgrades  being  approved)  and  for  evaluating  how  the  rotor  will  affect  the  environment 
(such  as  noise).  The  analyst  should  then  create  5  subgroups  from  the  taxonomy  for  each  of  the  previously  mentioned 
taxonomies.  Then,  the  analyst  would  use  a  technology-technique  selection  process  described  earlier  within  each 
subgroup,  resulting  in  a  multi-dimensional  forecast  to  predict  a  relevant  ecosystem  for  the  technology. 

In  Case  B,  the  VTOL  short-haul  vehicle,  forecasters  need  to  consider  many  more  dimensions  given  the  significant 
impact  of  such  a  vehicle  on  society.  Almost  all  dimensions  given  by  Martino  should  be  considered  with  a  high  degree 
of  importance.  Unforeseen  issues  in  the  business  case  for  this  vehicle,  difficulties  with  program  management, 
difficulties  in  navigating  safety  requirements  for  transport  aircraft,  acceptance  of  the  vehicle  in  society  and  integration 
into  culture,  and  concerns  over  fuel  emissions  and  noise  issues  could  compromise  the  project. 
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Both  cases  are  complex  systems,  but  need  different  forecasting  dimensions  and  techniques.  However,  forecasting 
in  many  dimensions  will  not  remove  the  uncertainty  surrounding  a  potential  technology  nor  guarantee  its  success.  The 
value  of  multi-dimensional  forecasting  is  not  in  the  improved  accuracy  of  the  prediction,  but  the  additional  knowledge 
that  decision-makers  have  about  the  technology. 

Future  research  should  include  examples  using  several  different,  real  complex  system  technologies  to  refine 
selection  methodology  for  forecasting  an  appropriate  ecosystem  surrounding  the  technologies.  In  this  endeavor, 
researchers  will  benefit  by  working  with  experts  to  reach  a  consensus  on  relevant  forecasting  dimensions  for  complex 
systems.  Researchers  can  then  work  to  match  these  dimensions  with  the  expected  results  and  family  characteristics  of 
the  forecasting  techniques. 


VII.  Conclusion 

This  study  surveyed  the  field  of  technology  forecasting  by  looking  at  both  previous  literature  surveys  and  text 
mining  academic  literature  to  identify  60  unique  technology  forecasting  techniques  and  associated  variations.  The  text 
mining  demonstration  illustrates  the  ability  of  the  technique  to  find  frontiers,  which  is  applicable  in  any  field  of 
research.  The  demonstration  also  provided  valuable  lessons  for  researchers  interested  in  using  text  mining.  The 
literature  associated  with  each  technique  was  reviewed  to  place  it  into  a  family,  describe  whether  it  was  quantitative 
or  qualitative  in  nature,  indicate  whether  it  could  be  used  for  explorative  or  normative  forecasting,  rate  12  criteria,  and 
characterize  the  expected  results  of  the  technique.  This  resulted  in  a  comprehensive  technology  forecasting  taxonomy. 
This  taxonomy  can  use  considerations  about  the  purpose  of  a  technology  forecast,  the  characteristics  of  the  technology, 
and  the  amount  of  expendable  effort  and  resources  to  select  an  appropriate  forecasting  technique.  Additionally,  this 
taxonomy  provides  a  valuable  resource  for  awareness  of  technology  forecasting  techniques.  For  future  work  on  the 
forecasting  technique  taxonomy,  the  criteria  ratings  and  characteristics  should  be  vetted  by  technology  forecasting 
experts  to  remove  any  controversy  in  its  use.  Next,  a  framework  for  selecting  forecasting  dimensions  followed  by 
technique  selection  within  each  dimension  should  be  built  upon  the  technique  taxonomy.  These  improvements  will 
increase  the  simplicity  and  efficiency  of  technology  forecasting  studies  for  complex  systems. 
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Appendix 


Table  4.  Complex  System  Technology  Forecasting  Taxonomy 


Method  [variations] 
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Expected  Results  for 
Complex  Systems 


References 


Action  [options] 
analysis 


V  aluing/ decision/  e 
conomic 


Both 


0.5 


0.5 


Economic  value, 
Decision 


Agent  modeling 
[Brownian  agents] 


Modeling  and 
Simulation 


Ex 


0.5 


0.5 


Behavior 


41  Ch.  31,42,  43,44, 
45 


Analogies 


Descriptive  and 
matrices 


H/S 


Ex 


Technology 

Development 


46 


Analytic  Hierarchy 
Process  (AHP) 


V  aluing/ decision/  e 
conomic 


N 


0.5 


Decision 


47 


Artificial  Intelligence 
[Machine  Learning] 


Statistical 


H/S 


Ex 


Metric  value 


38,39 


Artificial  Neural 
Network  [Adaptive 
neuro-fuzzy 
inference] 


Modeling  and 
Simulation 


Ex 


0.5 


0.5 


Metric  estimates 


7,  48, 49 


Backcasting 

[Obsolescence 

forecasting] 


Descriptive  and 
matrices 


N 


0.5 


Roadmap  to  reach 
scenario 


7,  50 


Bibliometrics  [research 
profiling,  patent 
analysis,  text  mining, 
citation  network 
analysis] 


Monitoring  and 
Intelligence/Statis 
tical 


H/S 


Ex 


Technology 

development, 

Alternatives 


41  Ch.  3,51,52,  53 
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Brainstorming 
[brainwriting, 
nominal  group 
process  (NGP)] 

Creativity 

s 

Both 

1 

1 

1 

1 

1 

0 

0 

1 

0 

1 

0 

0.5 

Alternatives,  expert 
judgements 

54 

Causal  Layered 

Analysis 

Creativity 

s 

N 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

Behavior 

50 

Causal  Models 

Modeling  and 
Simulation 

H 

Ex 

1 

1 

1 

0 

1 

1 

1 

1 

0 

0 

0.5 

0.5 

Metric  value 

8 

Checklists  for  Impact 
Identification 

Descriptive  and 
matrices 

s 

Ex 

1 

1 

1 

0 

1 

0 

0 

0 

0 

1 

0 

0.5 

Impact  Identification, 
Metric  estimates 

55 

Collaborative 
[Prediction  Markets, 
Online  Forecasting 
Communities] 

Expert  Opinion 

H/S 

Ex 

1 

1 

1 

1 

1 

0 

0 

1 

0 

0.5 

0 

0.5 

Any 

7,  50 

Complex  Adaptive 

System  modeling 
(CAS)  [Chaos] 

Modeling  and 
Simulation 

H 

Ex 

1 

0 

0 

0 

0.5 

1 

1 

1 

1 

0 

1 

1 

Technology 

Development 

56,  57,  58 

Correlation  Analysis 

Statistical 

H 

Ex 

1 

0 

1 

0 

0.5 

1 

0.5 

1 

0.5 

0 

0.5 

0 

Metric  value 

8 

Cost-benefit  Analysis 
[Monetized  and 
other] 

V  aluing/ decision/  e 
conomic 

H 

Ex 

0 

0 

0 

0 

1 

0.5 

1 

0 

1 

0.5 

0 

0 

Economic  value, 
Decision 

59 

Creativity  workshops 
[future  workshops] 

Creativity 

S 

Both 

1 

1 

1 

1 

1 

0 

0 

1 

0 

1 

0 

0.5 

Alternatives,  expert 
judgements 

60 

Cross-impact  Analysis 

Modeling  and 
Simulation  / 
Statistical 

H/S 

Ex 

1 

1 

1 

0 

1 

0 

0 

1 

0 

0.5 

0 

0.5 

Probabilities  of 
outcomes 

41  Ch.  9,  61 

Decision  analysis 
[utility  analysis, 
influence  diagrams, 
decision  trees] 

V  aluing/  decision/  e 
conomic 

S 

Both 

0 

0 

0 

0 

1 

1 

0 

1 

0 

0.5 

0 

0 

Probabilities  of 
outcomes,  Utility 
value 

7,  62 

Delphi  [iterative 
survey] 

Expert  Opinion 

S 

Both 

1 

1 

1 

1 

1 

0 

0 

1 

0 

1 

0 

0.5 

Any 

41  Ch.  4,  63 

Demographics 

Statistical 

H 

Ex 

0 

0 

0 

0 

1 

1 

1 

0 

1 

0 

0 

0.5 

Acceptability 

Diffusion  modeling 

Modeling  and 
Simulation 

H 

Ex 

1 

1 

1 

0 

1 

0.5 

0.5 

1 

0 

0.5 

0 

0 

Acceptability  (rate) 

64 

Economic  base 
modeling  [input- 
output  analysis] 

Modeling  and 
Simulation  / 

V  aluing/  decision/ 
economic 

H 

Ex 

0 

0 

0 

0 

0.5 

0.5 

1 

1 

1 

0 

0.5 

0 

Economic  trends, 
acceptability 

65 

Field  anomaly 
relaxation  method 
(FAR) 

Scenarios 

S 

Both 

1 

1 

1 

0 

1 

0 

0 

1 

0 

0.5 

0 

0.5 

Behavior 

41  Ch.  30,  66 

Focus  groups  [panels, 
workshops] 

Expert  Opinion 

S 

Both 

1 

1 

1 

1 

1 

0 

0 

1 

0 

1 

0 

0.5 

Any 

41  Ch.  23 

Fuzzy  Cognitive  Map 

Statistical  /  Expert 
Opinion  / 

Scenarios 

H/S 

Ex 

1 

0 

1 

0 

1 

0.5 

0 

1 

1 

0.5 

0.5 

0.5 

Behavior 

40 
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Heuristic 

Modeling  and 
Simulation 

H 

Both 

1 

1 

1 

1 

1 

0.5 

0.5 

1 

0 

0.5 

1 

0 

Metric  value, 
behavior 

50,67 

Hybrid  models 

- 

H/S 

Both 

1 

1 

1 

1 

1 

0.5 

0.5 

1 

0 

0.5 

1 

1 

Any 

68,  69,  70,71 

Innovation  system 
modeling 

Descriptive  and 
matrices 

S 

Ex 

0 

0 

0 

0 

0.5 

1 

1 

1 

0 

0.5 

0.5 

0.5 

Technology 

Development 

72,  73,  74 

Institutional  analysis 

Descriptive  and 
matrices 

S 

Ex 

1 

0 

1 

0 

1 

0 

0 

1 

0 

0 

0 

0.5 

Metric  values 

61 

Interviews 

Expert  Opinion 

S 

Both 

1 

1 

1 

1 

1 

0 

0 

1 

0 

1 

0 

0.5 

Any 

Long  wave  analysis 

Trend 

H 

Ex 

0 

1 

0 

0 

1 

1 

1 

1 

1 

0 

0 

0.5 

Technology 

Development 

75,76 

Mitigation  analysis 

Descriptive  and 
matrices 

S 

N 

0 

0 

0 

0 

0.5 

0 

0 

0 

1 

1 

0 

0.5 

Obsolescence 

scenarios 

Monitoring 
[environmental 
scanning,  technology 
watch] 

Monitoring  and 
Intelligence  / 
Statistical 

S 

Ex 

0 

1 

0.5 

0 

1 

1 

1 

1 

0 

0 

0.5 

0.5 

Technology 

Development 

41  Ch.  2,77 

Morphological  analysis 

Descriptive  and 
matrices 

s 

Both 

0 

1 

1 

0 

1 

0 

0 

0 

0 

1 

0 

0 

Alternatives 

78,  79 

Multi-Criteria  Decision 
Analysis  [data 
envelopment  analysis 
(DEA)] 

H 

N 

1 

0 

0 

0 

0 

0.5 

0.5 

0 

1 

0.5 

0 

0 

Alternatives,  Decision 

80 

Multiple  perspectives 
assessment 

Descriptive  and 
matrices 

s 

Both 

1 

0 

0 

0 

1 

0.5 

0.5 

0 

1 

0.5 

0 

0.5 

Behavior 

41  Ch.  33,81 

Organizational  analysis 

Descriptive  and 
matrices 

s 

Ex 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0.5 

0 

Technology 

Development, 

Behavior 

82 

Participatory  analysis 

Expert  Opinion 

s 

N 

1 

0 

1 

0 

1 

0 

0 

0 

1 

1 

0 

0.5 

Any 

41  Ch.  23,  83,  84 

Precursor  analysis 

Trend 

H 

Ex 

1 

0 

0 

0 

0 

1 

1 

1 

1 

0 

0.5 

0 

Technology 

development 

8 

Relevance  Trees 
[futures  wheel,  future 
polygon] 

Descriptive  and 
matrices  / 

V  aluing/ decision/ 
economic 

s 

Both 

0 

0 

0 

0 

1 

0 

0 

0 

1 

1 

0 

0 

Behavior 

41  Ch.  6-7  and  18,  85 

Requirements  analysis 
[needs  analysis, 
attribute  X 
technology  matrix] 

Descriptive  and 
matrices  / 

V  aluing/ decision/ 
economic 

H/S 

N 

0 

0 

0 

0 

0 

0.5 

0.5 

1 

0 

1 

0 

0 

Technology 
Development,  Metric 
values 

Risk  analysis 

Descriptive  and 
matrices  / 

Statistical 

H/S 

Both 

1 

1 

1 

0 

1 

0 

0 

1 

0 

1 

0.5 

0.5 

Any,  Probabilities  of 
outcomes 

86,  87 

Roadmapping 

[product-technology 

roadmapping] 

Descriptive  and 
matrices 

H/S 

Both 

1 

0 

0 

1 

0.5 

1 

0 

1 

1 

1 

0 

0.5 

Technology 

Development 

88,  89,  90 

13 

Aerospace  System  Design  Laboratory 
School  of  Aerospace  Engineering,  Georgia  Institute  of  Technology 


AE8900  MAV 


SPRING  2016 


Andrew  Smith 


Scenarios  [Scenarios 
with  consistency 
checks,  Scenario 
management] 

Scenarios 

H/S 

Both 

0 

1 

1 

0 

1 

0 

0 

0 

0 

1 

0 

0 

Alternatives 

41  Ch.  19  and  21,  91, 
92,93 

Scenario-simulation 
[gaming,  interactive 
scenario] 

Scenarios  / 

Modeling  and 
Simulation 

S 

Both 

0 

1 

1 

0 

1 

0 

0 

0 

0 

1 

0.5 

0.5 

Alternatives 

41  Ch.  24,  94 

Science  fiction  analysis 

Creativity 

S 

N 

0 

1 

1 

0 

1 

0 

0 

0 

0 

1 

0 

0.5 

Alternatives 

Social  impact 
assessment 
[socioeconomic 
impact  assessment] 

Descriptive  and 
matrices 

S 

Both 

0 

0 

0 

0 

1 

1 

0 

0 

1 

1 

0 

0.5 

Acceptability 

95 

Stakeholder  analysis 
[policy  capture, 
assumptional 
analysis] 

Descriptive  and 
matrices  / 

V  aluing/ decision/ 
economic 

S 

N 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

Metrics,  Knowledge 

96,  97 

State  of  the  future 
index  (SOFI) 

Descriptive  and 
matrices 

H/S 

Both 

0 

0 

0 

0 

1 

1 

0.5 

0 

0 

0.5 

0 

0.5 

Acceptability 

41  Ch.  37 

Sustainability  analysis 
[life  cycle  analysis] 

Descriptive  and 
matrices  / 

Modeling  and 
Simulation 

H 

Ex 

1 

0 

0 

1 

1 

1 

0.5 

0 

1 

0.5 

0.5 

0.5 

Alternatives,  Metric 
values 

98 

Systems  simulation 
[system  dynamics, 
KSIM] 

Modeling  and 
Simulation 

H 

Ex 

1 

0 

1 

0 

0.5 

1 

1 

1 

0 

0 

0.5 

0.5 

Metric  values 

41  Ch.  24,99,  100, 

101 

Tech  Sequence 
analysis  [Project 
evaluation  and  review 
Technique} 

Statistical  /  Expert 
Opinion 

H/S 

Ex 

0 

0 

0 

0 

1 

0.5 

0 

1 

0 

1 

0.5 

0.5 

Technology 

Development 

50 

Technological 

substitution 

Modeling  and 
Simulation 

H 

Ex 

1 

1 

1 

0 

1 

1 

1 

1 

0.5 

0 

0 

0.5 

Technology 

Development 

102,  103,  104 

Technology  assessment 

Descriptive  and 
matrices  / 

Modeling  and 
Simulation 

H/S 

Ex 

1 

1 

1 

1 

1 

1 

1 

0 

0 

1 

0.5 

0.5 

Metric  values, 
Acceptability 

61 

Trend  extrapolation 
[growth  curve  fitting 
and  projection] 

Trend 

H 

Ex 

1 

0 

0 

0 

0 

1 

1 

1 

1 

0 

0 

0.5 

Metric  values 

8,  105,  106,  107 

Trend  impact  analysis 

Trend/Statistical 

H 

Both 

1 

0 

1 

0 

0 

1 

1 

1 

1 

1 

0 

0 

Metric  values 

41  Ch.  8 

TRIZ  [patterns  of 
evolution,  Function] 

Creativity 

H 

Both 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0.5 

0.5 

Alternatives 

30,  108,  109,  110 

Vision  generation 

Creativity 

S 

Both 

0 

0 

0 

0 

0.5 

0 

0 

0 

0 

1 

0 

0.5 

Alternatives 

Wild  Cards 

Creativity 

S 

N 

0 

1 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

Alternatives 

7,  50 
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